Raise NoMethodError when installing Active Record with Rdoc 6.0.0

[y-yagi]

RDoc version

6.0.0

Steps to reproduce

Install Active Record with document generating.

$ gem install activerecord
Successfully installed activerecord-5.1.4
Parsing documentation for activerecord-5.1.4

RDoc::Parser::Ruby failure around line 1 of
lib/rails/generators/active_record/migration/templates/create_table_migration.rb

Before reporting this, could you check that the file you're documenting
has proper syntax:

  /home/yaginuma/.rbenv/versions/2.3.1/bin/ruby -c lib/rails/generators/active_record/migration/templates/create_table_migration.rb

RDoc is not a full Ruby parser and will fail when fed invalid ruby programs.

The internal error was:

	(NoMethodError) undefined method `[]' for nil:NilClass

ERROR:  While executing gem ... (NoMethodError)
    undefined method `[]' for nil:NilClass 

Cause

There are template files of Thor gem that extension is rb in Active Record. It seems that it is an error trying to parse it.

Since we have changed the extension of templates files in rails/rails@8dd76a7, this problem does not happen in Rails 5.2.0.
But old versions have this problem.

[aycabta]

@y-yagi OK, I think it's critical. I'm fixing it. Please wait for it.

[aycabta]

@y-yagi I got the picture. This issue is not RDoc's. I want to fix activerecord gem 5.1.4 because template files of 5.2 are renamed. How can I send a patch to 5.1.x?

[matthewd]

A change in a future 5.1.x release won't fix the problem for people who need to install older versions.

Those gems can be installed when using RDoc 5, but have the above error when RDoc 6 is present. Isn't that a regression?

[aycabta]

RDoc 5.x uses legacy Ruby lexical analyzer from IRB what was written 20 years ago. It has so many bugs. RDoc 5 tries to parse template files of Rails and finishes with no errors but it is a complex bug of legacy lexical analyzer and parser.

RDoc 6 uses Ripper. It fails to parse template files of Rails because it's not correct as Ruby syntax.

I think ActiveRecord 5.1.x needs the patch below.

--- a/activerecord/activerecord.gemspec
+++ b/activerecord/activerecord.gemspec
@@ -19,7 +19,7 @@
   s.require_path = "lib"

   s.extra_rdoc_files = %w(README.rdoc)
-  s.rdoc_options.concat ["--main",  "README.rdoc"]
+  s.rdoc_options.concat ["--main", "README.rdoc", "--exclude", "lib/rails/generators/.*/templates/.*/?.*\\.rb"]

   s.metadata = {
     "source_code_uri" => "https://github.com/rails/rails/tree/v#{version}/activerecord",

[aycabta]

Just in case I say. Ruby 2.5.0 contains RDoc 6.0.0. So gem install activerecord -v 5.1.4 and gem install activerecord -v 5.0.6 always fail on Ruby 2.5.

It's correct behavior because ERB files are illegal syntax as Ruby code.

[y-yagi]

Although we can deal with new versions, however, problems such as 5.1.2 will still be encountered when installing older specific versions.

[matthewd]

It fails to parse template files of Rails because it's not correct as Ruby syntax

I understand why it's not able to be parsed. And I'm personally very excited that RDoc has switched to a more modern proper parser. But it was previously not causing a fatal error on installation, so that is a regression. For RDoc 6 to be compatible, I think it must ignore the error, skip the bad file, and continue.

This error makes it impossible to install all Rails releases from the last 10 years. That is a big deal.

[aycabta]

This error makes it impossible to install all Rails releases from the last 10 years. That is a big deal.

OK, I understand what you think. I think that it's very important thing too. I'll try to implement detection of ERB file as soon as possible for 2.5, please wait.

[hsbt]

@matthewd I will release rdoc-6.0.1 with #573 . Can you fix a rails side both? #573 is completely workaround.