Program Details

1. Every 250ms the program reads a frame from the video feed.
2. This frame will be added to a buffer containing the last 16 frames.
3. On a separate thread is the following processing:
4. If a black screen is detected:
   1. Check the buffer for a level screen.
      • Run the reference image and current frame through OpenCv.AbsDiff()
      • Take the resultant image and run it through a OpenCv.ThresholdBinary() filter.
      • Take the resultant image and count the non-blank pixels.
      • Divide the non-blank pixels by the total pixels in the frame.
      • Subtract this value from 1 to get the percentage the images match.
      • If the images are at least a 90% match:
        • Grab the level author, level code & level name screeb regions from the current frame based on predefined rectangles.
        • Run each of these through the OpenCv.GammaCorrect() and OpenCv.Resize() filters for better contrast and size. This gives better results for the OCR.
        • Pass these images to the Tesseract library to perform the OCR.
          • For the level code, use the whitelist command to help with accuracy: tessedit_char_whitelist ABCDEFGHJKLMNPQRSTUVWXYZ0123456789-
        • Write the results to a JSON output file.
        • Broadcast new level info thru the Websockets server.
   2. For each image frame in the buffer:
      • For each available template image: (death icons, exit level button, start over button)
        • Run the frame and template image through OpenCv.MatchTemplate().
        • If there is a match:
          • Broadcast this event thru the Websockets server.
5. If a clear screen is detected
   1. Delay, to wait until the clear details are displayed on screen.
   2. Determine if comments are on the clear screen - this affects where the buttons and clear time are read from.
      • For each available template image: (world record, first clear)
        • Run the clear screen frame and template image through OpenCv.MatchTemplate().
        • If there is a match:
          • Broadcast this event thru the Websockets server.
   3. Read the clear time from the screen via Tesseract OCR library.
      • Broadcast the clear event and clear time thru the Websockets server.