Overview of PoGather Utility

PoGather is a specialized tool designed to collect essential data from MSDP (Media Server Deduplication Pool) DeDupe Catalog, Image Catalog, and system configurations. This data is crucial for the imgchk utility, which uses it to identify and manage orphaned images in a storage pool. By running PoGather on customer systems, necessary information is gathered and compressed into a single file, ready for analysis and data confirmation using imgchk. This process ensures that data integrity and security are maintained throughout the data lifecycle.

Functionality of Imgchk Utility

The imgchk analysis utility processes the collected data to detect inconsistencies. If orphaned images are found, imgchk generates corrective scripts to remove these from the MSDP catalog and data store. The generated files, such as `Out_DD_ORPHANED_filedel.py` and `Out_DD_ORPHANED_bpstsinfo.sh/bat`, help in managing and cleaning the orphaned data. Ensuring these orphaned images are properly managed is crucial for maintaining the security and integrity of the storage environment.

Supported Environments PoGather can be executed on:

•  NetBackup Master/Media Servers version 7.7.x.x or higher
• NetBackup Appliances
• Containerized appliances

There are three versions of PoGather available:

• Linux Master/Media Servers/Appliances
• Windows Master/Media Servers
• NetBackup Flex Appliance

Usage Instructions

PoGather is intended solely for data collection on customer machines. It does not alter the system, NetBackup, or any data. Here’s how to run PoGather based on the operating system:

• Linux: “. /PoGather_Linux”
• Windows: “PoGather_x64.exe”
• Flex Appliance: “./PoGather_Flex”

PoGather must be executed with root or administrator privileges to access necessary NetBackup and MSDP commands. All versions are compiled code, eliminating the need for Perl.

Execution Best Practices

1. Directory Setup: Run PoGather from a temporary working directory to ensure all output files are contained within it. These files are typically archived into a single compressed tar file named `PoGather_<Hostname>_<yymmdd>_<hhmmss>.tgz`.
2. Primary and Media Servers: On primary servers, PoGather automatically collects storage pool, volume information, and NetBackup catalog data. On media servers, it gathers DeDupe Catalog image details and PO information.
3. File Naming: Files generated include the server’s host name for easy identification, especially in environments with multiple media servers or NetBackup domains.

Common Error and Resolution

• Error:  “Can’t get database path from content router section of /etc/pdregistry.cfg.”
• Cause: This typically occurs on primary servers without MSDP.
• Solution: Ignore this error if PoGather is running on a primary server. If it occurs on a storage server with MSDP, further investigation may be needed.

Imgchk Analysis

The imgchk utility compares backup image information from NetBackup with DeDupe catalog data on the MSDP data store, providing summary information and various reports. It also generates corrective scripts for identified issues, ensuring that the actions taken are secure and preserve data integrity.

Reclaiming Data Storage Space

In large, scalable cloud environments, efficient storage management is crucial. Orphaned images can occupy significant storage space, leading to increased costs and reduced performance. By using PoGather and imgchk, administrators can reclaim this wasted storage space in real-time, enhancing overall storage efficiency.

Real-Time Data Storage Management

PoGather and imgchk facilitate real-time data storage management by:

1. Immediate Detection: Quickly identifying orphaned images that no longer serve any purpose.
2. Automated Cleanup: Generating scripts to remove unnecessary data, freeing up valuable storage space.
3. Ongoing Monitoring: Regularly running these tools ensures continuous optimization of storage resources.

Importance in Cloud Environments

In cloud environments, where scalability and cost-efficiency are paramount, effective storage management is essential:

1. Cost Savings: By reclaiming unused storage space, organizations can reduce their cloud storage costs.
2. Performance Improvement: Freeing up storage resources enhances the performance of cloud-based applications and services.
3. Data Governance: Ensuring that only relevant and necessary data is stored aligns with data governance policies and compliance requirements.

Data Protection and Security Measures

1. Data Privacy: Ensure that all data collected by PoGather is securely handled and stored. Only authorized personnel should have access to the collected data and the generated reports.
2. Encryption: Consider encrypting the compressed tar file containing the output of PoGather to protect it during transfer and storage.
3. Access Control:  Implement strict access control policies to ensure that only authorized users can execute PoGather and imgchk utilities and access the generated files and reports.
4. Audit Logs: Maintain audit logs of all activities related to PoGather and imgchk execution. This helps in tracking any changes made and ensures accountability.
5. Verification: Before executing any corrective scripts generated by imgchk, thoroughly verify their actions to prevent unintended data loss. Always perform these operations in a controlled and secure environment.

Execution Environment

• Linux/Solaris: Perl v5.8 or later is required.
• Windows: The tool runs on a 64-bit system.
  

Before running imgchk, ensure accurate image data collection by using PoGather. Always verify that images are for the correct storage pool and data selections to prevent accidental data removal.

Conclusion

PoGather and imgchk are powerful tools for maintaining data integrity and security within NetBackup systems. By carefully following usage guidelines and execution best practices, these utilities help ensure accurate data management and prevent data loss due to orphaned images. With a focus on data protection and security, organizations can confidently use these tools to enhance their data management processes. In large-scale cloud environments, the ability to reclaim storage space in real-time is particularly valuable, leading to cost savings, improved performance, and better compliance with data governance policies.

GitHub Repository

For more information and to access the PoGather utility, visit the GitHub repository: PoGather Repository.

Additional Reading

1. NetBackup MSDP Best Practices
   
2. Data Protection and Security in Cloud Environments
   
3. Managing Orphaned Images in Backup Systems
   
4. Optimizing Storage Management in Large Scale Systems
   

These references provide further insights into best practices and methodologies for efficient and secure data management using PoGather and imgchk in NetBackup environments.

Nizam Khan
Senior Storage and Data Protection Solution Architect
Big Sky Global LLC
nizam.khan@bigskyglobal.net

Published by: Martin De Juan