mcp-debugging

Core Testing Loop

• Whenever you change Go code and want to test the new functionality, you MUST follow these steps:

  1. Restart the Service: Run the development restart script to rebuild and restart the muster service with your changes.

     ./scripts/dev-restart.sh
     

  2. Initial Log Check: Verify that the service started correctly by checking the logs.

     journalctl --user -u muster.service --no-pager |tail -n 50
     

  Look for any startup errors or unexpected messages.

• Test with mcp-debug:

  • Use the mcp-debug tools to interact with the running service and test your new functionality.
  • Start by listing the available tools to ensure your new features are registered.

    # Example: List all tools from the mcp-debug server
    mcp_mcp-debug_call_tool(name="core_mcpserver_list_tools", arguments={ "serverName": "mcp-debug" })
    

  • Then, call the specific tools related to your changes to validate their behavior.

    # Example: Call a specific tool
    mcp_mcp-debug_call_tool(name="your_new_tool", arguments={...})
    

• Final Log Check:

  • Inspect the logs again to see the output and any errors generated during your testing.

    journalctl --user -u muster.service --no-pager |tail -n 50
    

• Important Note

  • Always use --no-pager: When using journalctl or systemctl, you MUST include the --no-pager flag. Without it, the output will be piped to a pager program, and you will not receive a response in the terminal.

Debugging muster via mcp-debug

• mcp-debug provides core debugging tools to test and explore the functionality of muster exposed in its aggregated MCP server.
• Use mcp-debug tools as the primary debugging interface rather than manual inspection

mcp-debug Tools

• mcp_mcp-debug_list_tools: list all the tools exposed by the aggregated mcp server
• mcp_mcp-debug_describe_tool: describe a tool exposed by the aggregated mcp server
• mcp_mcp-debugcall_tool: execute a tool exposed by the aggregated mcp server

Aggregated MCP server

• The aggregated MCP server is exposed by muster. This is the main subject of development and debugging here.
• The aggregated MCP server exposes tools via the model-context-protocol that you can explore via mcp-debug
• The main functionality of muster is exposed through the tools that are prefixed with core_*
• The aggregated MCP server can be compiled and restarted via ./scripts/dev-restart.sh

Debugging Examples

• 1. Verify All Services Are Running: Start by checking the overall health of the system:

  mcp_mcp-debug_call_tool(name="core_service_list", arguments={})
  

• 1. Check MCP Server Registration: Verify that all expected MCP servers are properly registered with the aggregator:

  mcp_mcp-debug_call_tool(name="core_mcpserver_list", arguments={})
  

• 1. If a service shows as unhealthy:
  2. Check its detailed status: core_service_status
  3. Try restarting it: core_service_restart
  4. Check if its MCP client is attached in the service list
• 1. Workflow validation fails:
  • Use core_workflow_validate to check syntax
  • Ensure tool names exist (check with mcp_mcp-debug_list_tools)
  • Verify argument schemas match the tool requirements

Important Notes

• The tools with the prefix x_ are coming from the MCP servers that are managed by muster and then aggregated in its own MCP server. Please don't try to fix them. They are separate projects.
• Tool names from individual MCP servers get prefixed like this x_kubernetes_* for the kubernetes MCP server and x_prometheus_* for the prometheus MCP server

Debugging Approach

• Phase 1: System Health Verification

  mcp_mcp-debug_call_tool(name="core_mcpserver_list", arguments={})
  mcp_mcp-debug_call_tool(name="core_serviceclass_list", arguments={})
  mcp_mcp-debug_call_tool(name="core_workflow_list", arguments={})
  mcp_mcp-debug_call_tool(name="core_service_list", arguments={})
  

  Key Checks:
  • All MCP servers services show "health": "Healthy" and "state": "Running"
  • ServiceClasses show "available": true
  • No missingTools in ServiceClass definitions
• ServiceClass-Specific Testing

  # Test specific ServiceClass availability
  mcp_mcp-debug_call_tool(name="core_serviceclass_available", arguments={"name": "<serviceclass-name>"})
  
  # Test instantiation with safe parameters
  mcp_mcp-debug_call_tool(name="core_service_create", arguments={
  "serviceClassName": "<name>",
  "label": "debug-test",
  "parameters": {...}
  })
  

• Dependency Verification

  # Test underlying tools directly
  mcp_mcp-debug_call_tool(name="<required-tool>", arguments={...})
  
  # Verify connectivity to external systems
  

• Service Recovery

  # Restart failed services
  mcp_mcp-debug_call_tool(name="core_service_restart", arguments={"label": "<service-name>"})
  
  # Check detailed status
  mcp_mcp-debug_call_tool(name="core_service_status", arguments={"label": "<service-name>"})
  

• Service Instance Management

  # List and clean up test instances
  mcp_mcp-debug_call_tool(name="core_service_list", arguments={})
  mcp_mcp-debug_call_tool(name="core_service_delete", arguments={"instanceId": "debug-test"})