Leveraging AWS Athena for Effective Visualization and Migration in VMware-Exit Scenarios

By Saumil Shah, Piyush Jalan / Aug 29, 2024

Content

Introduction

Transitioning from on-premises VMware environments to AWS requires effective discovery and visualization of server data. AWS Application Discovery Service, combined with AWS Athena, provides a powerful solution for streamlining this migration process. This blog will guide you through installing the discovery agent on Linux and Windows servers, enabling Athena exports, creating workgroups, and running impactful queries for insightful data analysis.

Automating Discovery Agent Installation

On Linux Servers

  1. Download and Install the Agent:

    curl -O https://aws-discovery-agent.s3.amazonaws.com/latest/linux/installer/AD-AgentInstaller_latest.tar.gz tar -xzf AD-AgentInstaller_latest.tar.gz sudo bash install -r your-home-region -k aws-access-key-id -s aws-secret-access-key

  2. Activate the Agent:

    sudo systemctl start aws-discovery-daemon.service sudo systemctl status aws-discovery-daemon.service

Enabling Athena Exports

  1. Configure Data Export in Application Discovery Service:
    • In the AWS Management Console, navigate to AWS Migration Hub -> Discover -> Data Collectors -> Discovery agent.

    • Enable data export to S3 for Athena analysis
  2. Create an Athena Workgroup:
    • Open the Athena console.
    • Create a new workgroup to manage query execution.

Running Athena queries:

  • Navigate to Athena -> Queries -> select ‘application_discovery_service_database’ as DB.

Use-Case 1: Latest System Performance Data

Objective: Analyze the latest performance data of servers to understand resource utilization and capacity planning.

Query:

WITH LatestData AS (
    SELECT 
        "SP"."agent_id",
        "OS"."os_name" AS "OS Name",
        "OS"."os_version" AS "OS Version",
        "OS"."host_name" AS "Host Name",
        "SP"."total_num_cores" AS "Number of Cores",
        "SP"."total_num_cpus" AS "Number of CPU",
        "SP"."total_cpu_usage_pct" AS "CPU Percentage",
        "SP"."total_disk_size_in_gb" AS "Total Storage (GB)",
        "SP"."total_disk_free_size_in_gb" AS "Free Storage (GB)",
        ("SP"."total_disk_size_in_gb" - "SP"."total_disk_free_size_in_gb") AS "Used Storage",
        "SP"."total_ram_in_mb" AS "Total RAM (MB)",
        ("SP"."total_ram_in_mb" - "SP"."free_ram_in_mb") AS "Used RAM (MB)",
        "SP"."free_ram_in_mb" AS "Free RAM (MB)",
        "SP"."total_disk_read_ops_per_sec" AS "Disk Read IOPS",
        "SP"."total_disk_bytes_written_per_sec_in_kbps" AS "Disk Write IOPS",
        "SP"."total_network_bytes_read_per_sec_in_kbps" AS "Network Reads (kbps)",
        "SP"."total_network_bytes_written_per_sec_in_kbps" AS "Network Write (kbps)",
        "SP"."timestamp",
        ROW_NUMBER() OVER (PARTITION BY "SP"."agent_id" ORDER BY "SP"."timestamp" DESC) AS rn
    FROM 
        "sys_performance_agent" "SP"
    JOIN 
        "OS_INFO_agent" "OS"
    ON 
        "SP"."agent_id" = "OS"."agent_id"
)
SELECT
    "OS Name",
    "OS Version",
    "Host Name",
    "agent_id",
    "Number of Cores",
    "Number of CPU",
    "CPU Percentage",
    "Total Storage (GB)",
    "Free Storage (GB)",
    "Used Storage",
    "Total RAM (MB)",
    "Used RAM (MB)",
    "Free RAM (MB)",
    "Disk Read IOPS",
    "Disk Write IOPS",
    "Network Reads (kbps)",
    "Network Write (kbps)"
FROM 
    LatestData
WHERE 
    rn = 1
LIMIT 10;

Explanation: This query retrieves the most recent system performance data for each agent, providing insights into CPU, memory, disk, and network usage. This data is crucial for identifying resource bottlenecks and planning capacity upgrades during migration.

Use-Case 2: Process Information for Specific Agent IDs

Objective: Analyze the processes running on specific servers to ensure critical applications are performing as expected.

Query:

SELECT 
    account_number,
    agent_id,
    agent_assigned_process_id,
    is_system,
    name AS "Process Name",
    cmd_line AS "Command Line",
    path AS "Path",
    agent_provided_timestamp AS "Timestamp"
FROM 
    application_discovery_service_database.processes_agent
WHERE 
    agent_id IN ("o-1k1h59xcjyxb7e6hm", "o-3l364e89qla1dpq14")
ORDER BY 
    agent_provided_timestamp DESC
LIMIT 50;

Explanation: This query retrieves the latest process information for specified agent IDs. It provides details such as process name, command line, path, and resource usage, which are essential for monitoring application performance and ensuring smooth operation during the migration process.

Conclusion

AWS Athena, combined with Application Discovery Service, provides a robust framework for visualizing and managing large-scale migrations from VMware to AWS. By automating the discovery process and leveraging powerful query capabilities, organizations can gain deep insights into their infrastructure, ensuring a smooth and efficient transition to the cloud.

Main Logo
Rocket