Monthly Performance Meetings: 2022

[fcollonval]

Here is a record of the agenda/minutes for our monthly meeting. Everyone is welcome to join.

• When: Every 4 weeks Wednesday 10AM Pacific Time
• Where: jovyan Zoom

Let's avoid using this thread for discussion. If you'd like to discuss something in the minutes, open a separate issue and reference this thread.

Meeting Notes

• 26 Oct 2022
• 28 Sep 2022
• 3 Aug 2022
• 6 Jul 2022
• 8 Jun 2022
• 12 May 2022
• 13 Apr 2022
• 16 Mar 2022
• 16 Feb 2022
• 19 Jan 2022

[fcollonval]

19 Jan 2022 Monthly meeting

Attendees

Name	affiliation	github username
Frederic Collonval	QuantStack	@fcollonval
William Stein	CoCalc/SageMath	@williamstein
Marc Udoff	DeShaw
Paul Ivanov	Noteable	@ivanov

Notes

• Marc

  • Limit outputs work done
    It will probably become open-source in the near future
    Make long notebook more manageable

• Frederic

  • Search extension: PR - Rework to carry search on the model
    • Document decision done: like ignoring the link markdown
    • Keep the GenericSearch for not supported outputs
    • william: encourage you to document the choices related to the subtle things like links, math in markdown, ansi output messages. I would be thrilled if there was an official jupyter specification of what search should do, and then client implementors could try to follow that spec.
    • Output messages is by far the hardest part, due to lack of control over them.
    • Regular expressions supported? Yes.
    • Look at other tools like VSCode to see what they are doing

[fcollonval]

16 Feb 2022 Monthly meeting

Attendees

Name	affiliation	github username
Eric Charles		@echarles
Frederic Collonval	QuantStack	@fcollonval
William Stein	SageMath/CoCalc	@williamstein

Notes

• echarles

  • draft JEP datalayer/jupyter-ui#10

  • Tools for memory leak tracking

    • from gitter by Mike

    This looks interesting for debugging memory leaks: https://blogs.windows.com/msedgedev/2021/12/09/debug-memory-leaks-detached-elements-tool-devtools/

    • [nick] more API coming in webdriver to get access to the memory allocation: https://gist.github.com/nolanlawson/01e7358f682dde01e9cb27f777acf195
      Done some work on JupyterLite

  • Search feature: will work on making the search debouncer customizable and to add triggering the search on enter.

    Looking at what VS Code is doing: the search up to 1.63 was scoped to the notebook code source (and not showing in rendered markdown cell - the cell state was switched when the current search match was in a markdown cell). But since 1.64, they display search in the rendered markdown cells and they have an option to search in outputs. But they warn about performance issue, because in that case they will rendered the outputs prior to search in it.

  • Web workers
    Some resources from Nick:
    https://developers.google.com/web/tools/workbox
    https://github.com/GoogleChromeLabs/comlink/
    Add offline support jupyterlite/jupyterlite#137

• Frederic

  • Browser hangs for notebook with hundreds of MB in metadata property
    The browser hangs when a Notebook is opened that contains hundreds of MB of data in the metadata property. jupyterlab#12145

  Should the user issue be stored in the the benchmark repo or the core one

  • Starting debugger is slow
    Debugger is very slow to start (21 seconds) jupyterlab#12012

  • Scheduled benchmark tests
    After investigation no degredation was found. Seems we got unlucky with the CI machine allocated.
    Needs to split better the test cases

  • Back to performance work next week

    1. Finish search from model PR
    2. Update benchmark repo with those new cases (doc and CI)
    3. Rework ToC extension to be based on data model
    4. Virtual notebook

• William: vague remark...

  • I'm implementing a "collaborative whiteboard" version of Jupyter notebooks right now, which leads to interesting performance questions.
  • Collaborative Whiteboard and use of Slate *everywhere* (including Jupyter markdown cells) sagemathinc/cocalc#5674 (comment)

• Memory Leaks - maybe testing for in an automated way:

  • https://gist.github.com/nolanlawson/01e7358f682dde01e9cb27f777acf195

• Nick Bollweg: Making webworkers more usable, e.g., for search

  • https://github.com/GoogleChromeLabs/comlink/
  • https://developers.google.com/web/tools/workbox
  • Add offline support jupyterlite/jupyterlite#137
  • comments on subtle performance implications of using react...

[fcollonval]

16 March 2022

Attendees

Name	affiliation	github username
Frederic Collonval	QuantStack	@fcollonval
Eric Charles	Datalayer	@echarles
Marc Udoff	D. E. Shaw	@mlucool
William Stein	SageMath/CoCalc	@williamstein

Notes

• Frederic

  • Final version of search from model
    At the end the search on output is still on the HTML. To hard to figure out what will be in the view (after testing mainly on Markdown).

• Marc

  • ad blocker hit performance in Chrome really bad - we should document and shout out that information
  • Works by rtrg:
    • CSS containment discusstion in that comment - unfortunately not available in FireFox.
      Context: clicking on the context menu

  %%javascript
  function addCss(cssCode) {
      const style = document.createElement('style');
      style.innerHTML = cssCode;
      document.head.appendChild(style);
  }
  
  addCss(`
      .jp-NotebookPanel.jp-Document {
          will-change: transform;
          transform: none !important;
      }
      .jp-NotebookPanel.jp-Document.lm-Widget.lm-mod-hidden {
          transform: scaleX(0) !important;
          display: block !important;
      }
  `);
  
  // This uses experimental CSS that is not supported by all browsers.
  // By setting content-visibility: auto, when the cell is off-screen,
  // style checks are avoided. We set an intrinsic height so that
  // scrolling is not too jittery.
  addCss(`
      .jp-Cell {
          content-visibility: auto !important;
          contain-intrinsic-height: auto 70px;
      }
  `);
  

  The above shows great improvements.

  • Try hidding the panels with CSS containment for speeding-up tab switching. It makes rendering of new tabs slower but then switch faster.
  • jupyterlab-limit-output can help user for with those long outputs
    User can blow their browser by memory size. There may be a memory leak somewhere - but enable to isolate it for now.
  • Loading static assets Jupyter Server issue
    In Cocalc the assets are loaded using NGinx

• Eric

  • Work on the kernel: On kernel area, recent work on kernel message parsing is merged Protocol alignment jupyter-server/jupyter_server#657
  • Virtual deferred rendering:
    • renderCellOnIdle improvements by waiting for long enough idle time #12166
    • Working on a PR to block cell list edition when the notebook is not yet fully rendered.
    • Probably gonna revert the setting to not activate virtual rendering by default

[fcollonval]

16 April 2022

Attendees

Name	affiliation	github username
Frederic Collonval	QuantStack	@fcollonval
Marc Udoff	D. E. Shaw & Co.	@mlucool
William Stein	CoCalc	@williamstein

Notes

• Frederic

  • Windowing notebooks:
    • Table of content refactor PR
    • Next step windowing notebooks
  • React is working on reloading state of unmounted components for case like virtual rendering or tab switch: Adding Reusable State to StrictMode reactwg/react-18#19
  • CSS containment landed in Webkit 15.4: https://webkit.org/blog/12445/new-webkit-features-in-safari-15-4/#css

• William

  • I'm back to windowing for CoCalc because of examples/complaints from users, including this math-heavy example: https://nbviewer.org/url/archive.org/download/musing_math/Volume_Solutions_To_Snub_Dodecahedron.ipynb
    • Good be interesting to add a benchmark test on that one.
    • Using React Windowing and move to
  • https://virtuoso.dev/
  • https://github.com/sagemathinc/cocalc/pull/
  • planning to render iframes differently to handle: jupyter iframe state: make it work properly sagemathinc/cocalc#4727
  • Shared issues with state storage with Plotly charts and windowing library.
    • example in pyinstrument; works by shoving output into an iframe.

• Marc

  • Firefox vs. Chrome: major improvements for the user experience in Firefox; mainly linked to the hit tests in Chrome. See 2 order of magnitude improvements.

[fcollonval]

11 May 2022

Attendees

Name	affiliation	github username
Jason Grout	Databricks	@jasongrout
WilliamStein	CoCalc	@williamstein
Marc Udoff	D. E. Shaw & Co	@mlucool
Pete Blois	Google
Frederic Collonval	QuantStack	@fcollonval

Notes

• Frederic

  • Windowing notebooks PR
    • To deal with outputs internal state, the current envision solution is to keep the outputs in the DOM.
    • iframe does not support to being moved in the DOM tree.

• William

  • Windowing using Virtuoso is live in cocalc, and also used everywhere in cocalc that involves lists.
  • Solved the iframe state problem
    The solution used is to place those elements in the header element (Virtuoso feature) and move it at the right position
    when the cell is in the view port.
  • Also solved for ipywidgets - k3d at least.
  • [Pete] Do you deal with jumping to link within the document? It is not supported in Cocalc. (but should be...)

[fcollonval]

8 June 2022

Attendees

Name	affiliation	github username
Frederic Collonval	QuantStack	@fcollonval
Jason Grout	Databricks	@jasongrout
William Stein	CoCalc/SageMath	@williamstein

Notes

• Virtualization in CoCalc:

  • I finally got one user complaint about my windowing implementation. %%html blocks with styles... the style isn't global and disappears when the style block leaves the DOM.
  • Solution: scan the cells to move those in the header hidden cell to be available globally.
    • Currently styles are accumulated, and not updated as the html block changes. This does seem to satisfy users, though.
    • But, that could change, actually. Just an issue of time.
    • Other comment: i didn't worry about style blocks in markdown html... yet.

• Frederic

  • Windowing notebooks PR
    

  Drag-and-drop with copy: we should not detach the source cell
  What about the case of %%html
  What about RTC structure change like added cell
  Listing to DOM events to see if the iframe is pulled out when scrolling into a specific position

  • william UX question: how do you enable/disable windowing; thoughts about scope of this?
    Currently there is a single user settings for all documents.
    But it is propagated to all notebook documents. So it will be
    easy to set it per document (same pattern as line numbers e.g.).
    About storing it within the document (aka notebook metadata);
    I did not thought about it and I am a bit reluctant to store
    view parameters in the document model.
    [william] The windowing parameter is usually conditioned by
    the document content (e.g. the type of outputs used). So it
    would make sense to store it in the document.

  • william question 2: any relations/issues between RTC and your virtualization implimentation?
    It has not been tested in RTC context.

  • william: I didn't know about "scroll to input"...

  • Links for the browser supporting virtualization:

    • https://github.com/WICG/display-locking/
    • < virtual-scroller >: chrome no longer pursuing
    • Aria virtual content: Chrome no active development
    • Custom highlighting: chrome proposing

  • Memory leak benchmark
    Start using fuite
    Trying BLeak - failed to make it works

[fcollonval]

6 July 2022

Attendees

Name	affiliation	github username
Frederic Collonval	QuantStack	@fcollonval
William Stein	CoCalc/SageMath	@williamstein

Notes

• Frederic
  • Performance:
    • CodeMirror 6 alone will bring a great boost: see benchmark
  • Memory leaks:
    • CI job at Add memory leak job #100 using fuite
      fuite outputs does not always matches results obtained with the 3 snapshots technique for the leaking objects and the memory delta may be the opposite - I trusted more the objects and collections leak information than the memory size.
    • PRs:
      • Clear Title.changed signal
      • JupyterLab PRs: 12750, 12774 and 12777
        Trying to get them for the next JupyterLab 3.4.x patch release.

Discussion of how to deal with memory leaks happened.

What cocalc does is this. But WeakRef's can be even better...

export function close(obj: object, omit?: Set<string>): void {
  if (omit != null) {
    Object.keys(obj).forEach(function (key) {
      if (omit.has(key)) return;
      if (typeof obj[key] == "function") return;
      delete obj[key];
    });
  } else {
    Object.keys(obj).forEach(function (key) {
      if (typeof obj[key] == "function") return;
      delete obj[key];
    });
  }
}

This is similar to the disposable interface in Lumino. But with a more radical approach as it deletes all keys.

[fcollonval]

3 August 2022

Attendees

Name	affiliation	github username
Frederic Collonval	QuantStack	@fcollonval
A. T. Darian	QuantStack	@afshin
M Krassowski		@krassowski

Notes

• Darian
  • Lumino 2: Try to test passive event but it breaks the test.
    https://chromestatus.com/feature/5745543795965952
    Probably should be applied only to some specific actions.
  • https://developer.mozilla.org/en-US/docs/Web/API/EventTarget/addEventListener
    If not specified, defaults to false – except that in browsers other than Safari and Internet Explorer, defaults to true for the wheel, mousewheel, touchstart and touchmove events.
  • https://github.com/WICG/EventListenerOptions/blob/gh-pages/explainer.md

[fcollonval]

28 September 2022

Attendees

Name	affiliation	github username
Frederic Collonval	QuantStack	@fcollonval
M Krassowski		@krassowski
William Stein	CoCalc/SageMath	@williamstein

Notes

• Frederic
  • Windowing notebooks PR is merged
    It can be tested on the pre release pip install --pre jupyterlab
  • Revert using transform scale for switching tabs JupyterLab PR
    It was crashing the UI with tabs containing iframe (such as documentation tab) in Chrome-based browser. So this is now adviced to be used only on Firefox (not tested on webkit)
    To be released in 3.4.8
• Mike
  • looking at menu performance

[fcollonval]

26 October 2022

Attendees

Name	affiliation	github username
Frederic Collonval	QuantStack	@fcollonval
A. T. Darian	QuantStack	@afshin
M Krassowski	Quansight	@krassowski
William Stein	CoCalc/SageMath	@williamstein
Vidar T Fauske	JP Morgan Chase	@vidartf

Notes

• [Mike] UI benchmark tool: PR
• [Frederic] Benchmarks project is currently broken due to unsupported syntax (nullish assignment operator) in vega-lite by older typescript version
• [Darian] Note about MessageLoop internal schedule implementation in Lumino 2
  In Lumino 2, the messages are gonna be processed in micro-tasks (no more in macro-tasks like in Lumino 1 due to setImmediate or requestAnimationFrame usage). So in pathologic case, this can create infinite loop (e.g. if an widget update method calls itself recursively).
• [Vidar] Suggestion for adding more realistic benchmarking case with multiple large files open: Having other large notebooks open while tab switching #124
• [Mike]
  • Delayed CSS style loading
  • MathJax 4/MathML support in browsers
    • do we need server URL for MathJax