Skip to content

DebbieArita/sre-week-three-task

 
 

Repository files navigation

Task 1 Implement a "big red button" for UpCommerce by creating a bash script to monitor the Kubernetes deployment dedicated to the Swype microservice defined in swype-deployment.yml. The script should be written in the empty watcher.sh file in the task repo, and trigger if the pod restarts due to network failure more than three times. Here's an optional, pseudocode hint of the tasks your watcher.sh file should perform:

  1. Define Variables: Set the namespace, deployment name, and maximum number of restarts allowed before scaling down the deployment.
  2. Start a Loop: Begin an infinite loop that will continue until explicitly broken.
  3. Check Pod Restarts: Within the loop, use the kubectl get pods command to retrieve the number of restarts of the pod associated with the specified deployment in the specified namespace.
  4. Display Restart Count: Print the current number of restarts to the console.
  5. Check Restart Limit: Compare the current number of restarts with the maximum allowed number of restarts.
  6. Scale Down if Necessary: If the number of restarts is greater than the maximum allowed, print a message to the console, scale down the deployment to zero replicas using the kubectl scale command, and break the loop.
  7. Pause: If the number of restarts is not greater than the maximum allowed, pause the script for 60 seconds before the next check.
  8. Repeat: After the pause, the script goes back to step 3. This process repeats indefinitely until the number of restarts exceeds the maximum allowed, at which point the deployment is scaled down and the loop is broken.

Task2 Identify potential solutions or products, whether free or commercial, to address the toil in the ticketing system. These solutions should aim to mitigate issues such as recurring obsolete alerts and lack of prioritization.

Solutions to Address Toil in Ticketing Systems

To address issues such as recurring obsolete alerts and lack of prioritization in ticketing systems, consider the following solutions:

1. PagerDuty

  • Description: PagerDuty is an incident management platform that helps streamline and prioritize alerts, enabling teams to respond to critical issues efficiently.
  • Key Features:
    • Alert Prioritization: Allows setting up escalation policies and routing rules based on severity and impact, ensuring important alerts are addressed promptly.
    • Integration: Integrates with monitoring tools and ticketing systems to reduce redundant alerts and automate incident management workflows.
    • Incident Response: Provides on-call scheduling, mobile notifications, and incident response orchestration to improve team coordination and response times.
  • Pricing: Offers a free tier with basic features. Paid plans are available with advanced functionalities.

2. Opsgenie

  • Description: Opsgenie, now part of Atlassian, is an incident management and alerting solution designed to reduce alert fatigue and optimize incident response.
  • Key Features:
    • Alert Aggregation: Consolidates alerts from various monitoring tools and deduplicates redundant notifications to reduce noise.
    • Escalation Policies: Allows defining escalation rules based on alert severity, ensuring critical incidents are promptly addressed.
    • Collaboration Tools: Facilitates real-time collaboration among teams during incident response through chat, runbooks, and status updates.
  • Pricing: Offers a free plan for small teams. Paid plans provide additional features like on-call scheduling and integration options.

3. VictorOps

  • Description: VictorOps is an incident response and alerting platform that focuses on improving incident management processes and reducing operational toil.
  • Key Features:
    • Incident Lifecycle Management: Guides teams through the entire incident lifecycle, from detection and response to post-incident analysis.
    • Integration Hub: Integrates with various monitoring and ticketing systems, allowing for centralized alert management and automated incident creation.
    • Performance Analytics: Provides analytics and reporting tools to identify recurring issues and optimize incident response strategies.
  • Pricing: Offers a free trial. Pricing is based on the number of users and features required.

4. Freshservice

  • Description: Freshservice is a cloud-based IT service management (ITSM) software that includes ticketing, asset management, and incident prioritization capabilities.
  • Key Features:
    • Intelligent Ticketing: Automates ticket categorization and prioritization based on predefined rules and SLAs, reducing manual effort.
    • Asset Discovery: Automatically discovers and tracks IT assets, facilitating proactive maintenance and incident prevention.
    • SLA Management: Monitors and enforces SLA compliance, ensuring critical issues are addressed within specified timelines.
  • Pricing: Offers a free plan for small teams. Paid plans provide additional features like asset management and advanced reporting.

5. Zendesk

  • Description: Zendesk is a customer service and support platform that includes ticketing, workflow automation, and analytics tools to enhance incident management processes.
  • Key Features:
    • Automated Workflows: Enables automation of repetitive tasks and ticket routing based on priority and agent availability.
    • Multichannel Support: Integrates with various communication channels (e.g., email, chat, social media) for centralized ticket management.
    • Performance Metrics: Provides real-time analytics and dashboards to track ticket volumes, response times, and customer satisfaction metrics.
  • Pricing: Offers a free trial. Pricing varies based on the plan and features selected.

Conclusion

Implementing one of these solutions can significantly improve the efficiency and effectiveness of incident management processes within ticketing systems. Each platform offers unique features designed to address common pain points such as alert fatigue and prioritization challenges, ultimately reducing operational toil and enhancing overall team productivity. Choose the solution that best aligns with your organization's needs and budget constraints.

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 39.9%
  • Shell 38.8%
  • Dockerfile 21.3%