Put `*Sync` methods behind a flag in some future major version

[ChALkeR]

No, I am not writing this from a mental hospital, please read till the bottom.

The purpose of this is to discourage using *Sync versions of the methods that could be async.

I propose to put all or at least a part of *Sync methods behind a runtime flag in some future major version of io.js and to split documentation, moving all *Sync methods to a separate page. This of course would be semver-major.

Atm, there are *Sync methods defined in zlib, fs, child_process, crypto modules and additionaly used in repl and module modules.
To allow usage inside io.js itself (i.e. for require), they could be moved to «private» methods (beginning with a _ sign).

1. *Sync methods suggest bad practices (see 2), but when someone with full understanding of the consequences needs them, they could be constructed from userspace. See https://github.com/abbr/deasync, it creates synchronous methods from async methods.
   If deasync module is not good enough for this, this could be done when there is would be a good enough solution.

2. When a newcomer begins writing something using io.js, he or she goes to the documentation looking how to do something, sees *Sync methods without any warnings there and almost certanly begins with using them, because that's what he or she is used to. That's easier for a newcomer than spending a few minutes reading how he or she should actually do stuff. And don't blame the newcomer, it's the presense of *Sync methods in the documentation that suggests to him or her that it's an ok way to do things. This results in a big pile of bad code by the time when the person understands that it should be rewritten. And people don't like to rewrite code for no visible reson, leaving this code to be legacy (see 3).

3. When someone writes synchronous code (see 2) it limits how he or she can use async functions without rewriting most part of the logic, so he or she comes complaining about that there should be a *Sync version of everything out there (as the presense of *Sync versions in the core suggests it). See Is there a sync version encapsulating libmagic in node.js ? mscdex/mmmagic#32 and motivation behind https://github.com/abbr/deasync.

4. It's completely broken either way. Even for «simple one-time scripts». See zlib: memory leak with gunzipSync #1479 — a person was doing something like files.forEach( …zlib.gzipSync(…) …), in what I suppose was a simple script. What could possibly go wrong? Memory usage has gone completely bad in his script. And even manual calls to gc() do not help. Testcase:

   'use strict';
   
   var zlib = require('zlib');
   var data = 'abcdefghijklmnopqrstuvwxyz';
   var gzipped = zlib.gzipSync(data);
   
   function call() {
     var contents = zlib.gunzipSync(gzipped);
   }
   
   for (var i = 0; i < 100000; i++) {
      call();
      if (i % 1000 === 0) {
          gc();gc();gc();gc();
          console.log(i + ' ' + JSON.stringify(process.memoryUsage()));
      }
   }

5. People go to the doc, see *Sync versions (see 2), use them — then everyone are telling them that they are using io.js wrong just based on that fact: gripe: deprecating fs.exists/existsSync #1592 (comment).

   if you're using existsSync inside your server I'd argue you're not using io.js correctly to begin with

6. One can argue again that using *Sync versions of methods is ok in scripts and that that's simplier, but:

   1. See 4.
   2. For one-time, throw-away scripts you can turn that flag on.
   3. Promises-based code is as clean as *Sync-based code. Promises-based code with accurate error handling is much cleaner that *Sync-based code with accurate error handling. Using Promisify won't be needed once Promises go to the core (see Feature Request: Every async function returns Promise #11).

This will require all public modules that are using *Sync methods either to rewrite things using async methods or require/include something like https://github.com/abbr/deasync to be compatible.

This can be done separately for various core modules/methods. For example, zlib.*Sync are maybe the worst of them and it looks to me that they are not actively used in public modules.

[cjihrig]

I think synchronous methods do have their place, and shouldn't be moved behind a flag. You just shouldn't be using synchronous calls in hot code. require() itself is synchronous.

[vkurchatkin]

-1. Sync functions are very useful in various cases.

deasync is definitely the worst idea ever. 1. It doesn't actually make asynchronous calls blocking 2. It's inherently broken.

[ChALkeR]

@cjihrig I was not talking about require, I was talking about *Sync methods — synchoronous variants of those methods that are async by themselves.

@vkurchatkin Checked the code. Isn't it possible to create an actual lock instead of a loop?

By the way, zlib.*Sync seems to be almost unused in comparison with fs.*Sync. For example, in node_modules of the project I have currently opened, fs.*Sync is used in 636 files while zlib.*Sync is not used at all.

[vkurchatkin]

@ChALkeR I'm not sure how lock can help? To run async functions you need an event loop. It can't be the main loop, as unrelated requests would be processed as well. Something like this would work: nodejs/node-v0.x-archive#7323

[rlidwka]

+1

Only good use for sync functions is for one-line scripts, so adding a flag for it seems reasonable.

But usage of sync functions in any public modules should definitely be discouraged. Not only because of the event loop blocking, but because sync functions can't be replaced with async functions should the need for it arise.

For example, Jade uses readFileSync to read templates internally. What if I want to load them from a database instead? If it was an async readFile, I'd just monkey-patched require('fs'), but with readFileSync it just ain't happening.

Even synchronous require is a big pain in the ass if you want to require something from a remote http server or a zip archive instead of a filesystem. We've talked about that before.

I believe sync functions should be removed completely when ES7 async/await support lands in v8. Until then we should be gradually deprecating them.

[ChALkeR]

@vkurchatkin nodejs/node-v0.x-archive#7323 looks like a cleaner solution to me than the current deasync module.

Once again: I do not propose to remove *Sync just now, I am talking about masking them in some future major version, when Promises will be treated as first-class citizens etc. And this is not about all synchronous methods, it's about those methods that have a normal async version.

This might require a better way to build a sync version of an async function than the deasync module, if it is not good enough.

[jonathanong]

I'd rather have an async-only mode that warns/throws (an option for both). Creating CLI scripts that require runtime flags is a pain in the ass.

[piscisaureus]

I'm -1 on putting Sync methods behind a flag. There are plenty of valid use cases for them.
If anything, maybe we could add --trace-blocking that prints a message whenever the event loop is blocked on i/o.

[ChALkeR]

@jonathanong That woldn't do anything. It would not lower *Sync usage in modules, it wouldn't discourage newcomers from using *Sync methods, it wouldn't motivate people to port their legacy code to async. It wouldn't solve any problem highlighted in the inital post.

You could just grep for [a-z]*Sync\( if you want to just find all usages of them, there is no need for that to be traced.

[ChALkeR]

@piscisaureus They introduce problems because they seem to be an easy solution at a first glance but result in huge problems later (see 2, 3). Even for use cases that look like valid there might be problems (see 4). Such flag might be useful, but this is a different question from the one that I want to discuss here. A relevant thing would be to propose some other way of discouraging newcomers (and not-so-newcomers) from (mis-)using *Sync methods.

@piscisaureus, @vkurchatkin Please, list specific use cases for this to be constructive.

[ChALkeR]

Relevant: #5 (comment)

[vkurchatkin]

To name some: scripting, bootstrapping, logging, using IO inside synchronous code. Sometimes blocking is fine. async-await + promises would be nice, but it doesn't solve all the problems.

Also, point by point:

1 -

when someone with full understanding of the consequences needs them, they could be constructed from userspace

No, they couldn't

2 - let's fix the docs

3 -

When someone writes synchronous code (see 2) it limits how he or she can use async functions without rewriting most part of the logic

This point as against removing Sync functions, if I understand it right

4 - this looks bad, actually

5 - See 2

6 -

Promises-based code is as clean as *Sync-based code.

No, it's not.

[ChALkeR]

@vkurchatkin

1. when someone with full understanding of the consequences needs them, they could be constructed from userspace

   No, they couldn't

   By this (added some time ago, after you shared deasync considerations):

   If deasync module is not good enough for this, this could be done when there is would be a good enough solution.

   I meant that this whole issue would have to wait until that problem (constructing sync methods from async methods) is solved. When and if that problem would be solved, the answer would be «Yes, they could».

2. Yes, fixing the docs is part of the issue, it's mentioned at the very top. Doing only that would not solve the issue completely, but will work as a partial solution.

3. No, it's a point for discouraging being contaminated by *Sync at the first place and against making it look like every async function should have a sync variant. No one is going to do that in all the user-contributed modules, they are generally async-only (for stuff that should be async), without sync variants.

4. Yes. zlib.*Sync is bad, broken and not actively used, it should be the first candidate for deprecating IMO. But the same should be true for all heavy memory-using *Sync code.
   It's just an example for that you can not predict when sync code goes wild. Everything was made with garbage collection and async in mind.

5. See 2.

6. When Promises would be first-class citizens in io.js, it would be. And do not forget about error handling, if you take error handling into account, Promises-based code even now looks much cleaner that *Sync-based code. The latter would be a bunch of try-catches.