Feature/ha #207
Merged
Feature/ha #207
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Deleted multiple unused modules, including auth, badge, bruteforce, certificate, and others, to reduce complexity and improve maintainability. - Cleaned up associated test files and configurations to reflect the removal of these modules. - Updated Dockerfiles to ensure they align with the current project structure.
…ve unused MongoDB and SQL files - Updated the database connection logic in main.go to utilize the new infra package for SQL and MongoDB connections. - Deleted obsolete mongo.go and sql.go files to streamline the codebase and eliminate unused dependencies.
… struct - Introduced new fields for Redis configuration, including host, port, password, and database. - Added a field for queue concurrency to manage the number of concurrent workers processing tasks.
- Introduced new Makefile targets for building and running the producer binary. - Added `build-producer` to compile the producer service and `run-producer` to execute it. - Created an alias `producer` for convenience in running the producer service.
- Added Redis service configuration to Docker Compose files for development environments. - Implemented Redis event bus functionality, allowing for distributed event handling across services. - Updated various modules to utilize the new Redis event bus for event subscription and publishing. - Introduced helper functions for unmarshalling event payloads from Redis. - Refactored event handling logic to accommodate the new event bus implementation.
…the producer service - Deleted ARCHITECTURE.md, QUICKSTART.md, and README.md files as they contained outdated information regarding the producer service. - This cleanup aims to streamline documentation and prepare for future updates.
- Added Redis configuration options to the .env.dev.example file for local development. - Updated Docker Compose files to change Redis port from 6379 to 6381 for consistency across environments. - Introduced Asynq monitoring service in the Docker Compose configuration to facilitate task management and monitoring.
- Introduced a new development script for the worker in package.json to facilitate local development. - Updated turbo.json to include caching and persistence settings for the new worker script. - Refactored healthcheck methods to follow Go naming conventions by capitalizing the method names for better visibility and consistency.
- Added new Asynq client and inspector to the worker for improved task management. - Updated health check queue priorities to include 'healthcheck' and 'ingester' for better task handling. - Refactored health check methods to return structured results, including ping time and maintenance status. - Streamlined event listener methods for monitor events, preparing for future enhancements.
- Introduced new Makefile targets for building and running the ingester binary. - Updated package.json to include a development script for the ingester service. - Modified turbo.json to add caching and persistence settings for the ingester. - Enhanced queue processing in the worker to enqueue tasks for the ingester service.
- Added a new API service with a main entry point in cmd/api/main.go. - Updated package.json to change the development script for the API service. - Enhanced the health check processing by moving important logic to the ingester, including heartbeat management and notification handling. - Refactored health check methods to streamline event handling and integrate with the new queue system. - Updated .gitignore to exclude new worker and ingester directories.
- Updated the logging setup in both producer and worker services to filter out debug logs in development mode. - Customized the log output format to include timestamps and remove unnecessary keys for improved readability. - Adjusted queue concurrency setting in the configuration to increase the number of concurrent workers from 10 to 300. - Modified health check task scheduling to dynamically set timeout and retention based on monitor interval, improving task management. - Implemented stale task detection in the worker to skip processing of outdated health check tasks, enhancing efficiency.
…s configuration - Replaced cron-based scheduling with Asynq for managing periodic tasks in the MonitorScheduler. - Added a new ProvideAsynqScheduler function to create and configure the Asynq scheduler. - Updated Docker Compose to use a Redis Alpine image and added a volume for Redis data. - Enhanced health check task management by registering tasks with Asynq, improving reliability and performance. - Removed deprecated cron references and streamlined job management within the MonitorScheduler.
…ntenance status - Commented out the logic for checking if a monitor is under maintenance in the buildHealthCheckPayload method. - Set the IsUnderMaintenance field to false by default in the health check payload. - This change simplifies the health check process while maintaining the existing structure for potential future use.
…dependencies - Updated the logging setup in the ingester to filter out debug logs in production mode and customize the log output format. - Refactored the producer module by removing the MonitorScheduler and EventListener dependencies, simplifying the dependency injection process. - Improved the producer's scheduling logic by integrating Redis for managing monitor intervals and health check tasks. - Removed the deprecated monitor_scheduler.go file, consolidating scheduling functionality within the producer.
…functionality - Integrated leader election into the producer, allowing it to manage scheduling based on leadership status. - Updated the EventListener to handle monitor events more effectively, including JSON payload unmarshalling and leadership checks. - Refactored the producer's methods for adding, updating, and removing monitors, ensuring they interact with the new leader election logic. - Enhanced logging to provide better insights during event handling and leadership transitions.
…ling methods - Removed deprecated scheduling methods and constants related to monitor management, including leadership monitoring and scheduling initialization. - Streamlined the producer's responsibilities by focusing on essential functionalities, enhancing maintainability and readability. - Updated the import statements to reflect the changes in dependencies and removed unnecessary code artifacts.
… scheduling - Added functionality to check for existing scheduled monitors in Redis before scheduling new ones, improving efficiency. - Introduced a map for quick lookup of existing monitor IDs, reducing unnecessary scheduling operations. - Enhanced logging to provide insights on newly scheduled and already scheduled monitors during initialization.
- Updated the leader election logic to use a fresh context for cleanup when the original context is cancelled, preventing potential issues during leadership release. - Added a timeout to the cleanup context to ensure it completes within a specified duration, enhancing reliability during shutdown.
- Changed the maximum items to claim per tick from 1000 to 2 for better control over processing. - Renamed scheduling methods to reflect their purpose in monitor syncing. - Enhanced the runLeadershipMonitor method to manage monitor syncing instead of scheduling. - Updated processMonitor to return the monitor interval for rescheduling, improving task management. - Streamlined job processing initiation to start immediately upon producer start, ensuring efficient task handling.
…processing - Reintroduced and refined the logic for checking if a health check task is stale, ensuring tasks older than 1.5 times their interval are skipped. - Enhanced logging for skipped tasks to provide better insights into task management and processing delays.
…ing to prevent duplicates - Updated the push handler, producer, and worker modules to use EnqueueUnique for task scheduling, ensuring that duplicate tasks are not created. - Introduced unique keys based on monitor IDs and timestamps to enhance task management and processing reliability. - Adjusted the maximum items to claim per tick in the producer constants for improved control over processing.
…nhance health check logging - Commented out the logging statements for successful task enqueuing in the queue service to reduce verbosity. - Updated health check task processing to use Info level logging for enqueued results, improving clarity in task management. - Introduced duration logging for health check tasks to provide insights into processing times.
- Implemented the creation of an index on the monitor_id field in the maintenance collection to improve query performance. - Added error handling to panic if the index creation fails, ensuring immediate feedback on potential issues.
…odule - Increased the default queue concurrency from 300 to 1000 to improve task processing capabilities. - Introduced a new configuration parameter for producer concurrency, allowing up to 128 concurrent producer goroutines. - Adjusted batch claiming size to 50 for better distribution of tasks among workers, reducing contention. - Enhanced logging to include worker IDs for better traceability during task processing and error handling. - Implemented context management in task processing to ensure graceful handling of health check tasks.
…k logging - Modified Redis configuration to disable append-only mode and adjust save settings for improved performance. - Enhanced logging in the ingester task handler to include task processing duration, providing better insights into task execution times.
- Modify Makefile to streamline development environment startup with updated command structure. - Enhance package.json by adding individual development scripts for API, producer, worker, and ingester. - Refactor import paths in API key module files to use the internal directory structure. - Remove obsolete server binary file from the repository.
…ries - Enhanced the scheduling logic to retrieve existing scheduled monitors from both due and lease sets in Redis. - Implemented a mechanism to remove stale monitors that are no longer active in the database. - Updated logging to reflect the number of removed stale monitors and the total count of active monitors scheduled.
…emon execution - Added a delay of 2 seconds to the nodemon execution commands for api, producer, worker, and ingester to improve stability during development. - This change helps prevent potential race conditions when restarting the services.
…er modules - Removed direct dependencies on bun.DB and replaced with a unified GracefulDatabaseShutdown function to handle various database types. - Enhanced shutdown logic to support both SQL and MongoDB, ensuring proper disconnection and resource cleanup. - Updated logging for clearer shutdown status across modules.
Contributor
|
🚀 Release Candidate Built: |
Contributor
|
🚀 Release Candidate Built: |
…atabase name for improved clarity
…ed size and security
Contributor
|
🚀 Release Candidate Built: |
…proving health checks - Introduced Redis service in development and production Docker Compose files for better caching and message brokering. - Updated MongoDB and PostgreSQL services to use a unified 'database' naming convention. - Implemented health checks for Redis and other services to ensure proper startup and readiness. - Removed redundant environment variables related to Redis from service definitions, streamlining configuration.
…ation - Changed DB_HOST in the production environment example from 'localhost' to 'database' for better deployment alignment. - Updated Redis port configuration in development Docker Compose files to use an environment variable for flexibility. - Added Redis environment variables in the Playwright workflow for improved testing setup.
…st for local testing
…ations for improved deployment consistency
…figurations for consistency
…abase and server configurations for improved deployment consistency
…database' for consistency with naming conventions
Contributor
|
🚀 Release Candidate Built: |
…es, ensuring the most recent versions are deployed.
…rom scratch to alpine for improved size and security
Contributor
|
🚀 Release Candidate Built: |
…rvices - Adjusted health check intervals to 30s, timeouts to 2s, and retries to 5 for improved reliability. - Added start_period of 5s to ensure services have adequate time to initialize before health checks commence.
…ironment variable for flexibility
…er Compose files - Updated health check commands to use a more structured format for clarity. - Standardized health check intervals, timeouts, and retries across all services to enhance reliability. - Renamed services from 'mongodb' and 'postgres' to 'database' for consistency. - Introduced new services (redis, api, producer, worker, ingester) with appropriate health checks and dependencies to improve service orchestration.
Contributor
|
🚀 Release Candidate Built: |
…MongoDB and PostgreSQL to 'database' for consistency
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
…anced system availability and scalability, including updates for microservices configuration in Docker Compose.
|
|
||
| if err != nil { | ||
| panic("Failed to create index on stat_minutely collection:" + err.Error()) | ||
| } |
There was a problem hiding this comment.
Bug: Duplicate Index Creation Causes Overhead
The NewMongoRepository function creates a duplicate index on (monitor_id, timestamp) for the stat_minutely collection. This redundant operation could lead to unnecessary overhead or MongoDB warnings.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
partly solves #175