Parallelize Producers #12
Description
Currently, we are running with one Producer at a time. While this is fine for most debug jobs, it is not sufficient in production. The main bottleneck is the I/O time when reading alerts on disk before they are ingested in the Kafka cluster (producer rate is bounded to ~5 MB/s, which corresponds to I/O operations).
Action item: split the list of alerts to be read in chunks, and instantiate several producers - one per chunk.