allow custom consensus in indexer

[scholtz]

Summary

Following pull request #1041 because of merge conflict i have implemented it once again.

Algorand indexer at the moment does not allow using custom consensus on private networks. This pull request adds this feature.

Feel free to customize it and please merge asap. The last pull request from june had already a lot of merge conflicts and it is basically 10 lines of code. Thank you

Test Plan

I tested it manually in aramid chain (custom consensus) indexer.

Build:
https://github.com/scholtz/AlgorandNodes/blob/main/docker/aramid-indexer-1.14.2.sh
https://github.com/scholtz/AlgorandNodes/blob/main/docker/compose-indexer.dockerfile

Deploy:
https://github.com/scholtz/AlgorandNodes/blob/main/kubernetes/aramid-main-indexer-de/deployment.yaml

[winder]

Hi @scholtz, we're in the process of removing go-algorand as a dependency for Indexer. It will probably still be another couple of months before we finish. Once the dependency is removed Indexer would use an algod process, so you could manage the consensus settings there as usual.

Could you help me understand how this change would impact you? Basically, would this require you to run a patched version of Indexer or would it also require your users to run a patched version of indexer?

The next Indexer release will include our conduit tool, which has a lot of changes and would require slight modification to this PR. #1326

A good spot for this functionality would be here, which would make it easy to remove along with the block processor when the time comes:
https://github.com/algorand/indexer/blob/conduit/conduit/plugins/processors/blockprocessor/block_processor.go#L80

[scholtz]

hi @winder

I am not able to run the indexer with custom consensus even with version 2.15.0-rc2.

{"error":"MakeProcessorWithLedgerInit() err: InitializeLedger() simple catchup err: RunMigration() err: MakeProcessor() err: MakeProcessor() err: MakeProcessor() err: unsupported protocol aramidmain-v1.0/1","level":"error","msg":"blockprocessor.MakeProcessor() err MakeProcessorWithLedgerInit() err: InitializeLedger() simple catchup err: RunMigration() err: MakeProcessor() err: MakeProcessor() err: MakeProcessor() err: unsupported protocol aramidmain-v1.0/1","time":"2022-11-16T15:13:51Z"}

The use case is for example CDBC who runs the private network with custom consensus file of the algod.

Or we run the custom conensus to increase note field length because we need it and we like algo tech.

This change is impacting our product that we cannot use the official algo software, but we have to use "patched" version of the algorand software. We would like to help the community for example cdbc banks to be able to use the algo software properly.

It is just 10 lines of code.. can you somehow incorporate it there, and in half year solve the thing to split it up from the algod?

[scholtz]

For the record it is just 3 additions to the code ... i have built a fixed docker image using this build script: https://github.com/scholtz/AlgorandNodes/blob/1e434c314fc813554881388e6ea8b9752a00753f/docker/compose-indexer-fix.dockerfile#L20

[algoanne]

Hi @scholtz thanks for the PR - we agree that supporting custom consensus would be useful and we'd like to support it. But, we won't be able to do that until March at the earliest because of the significant refactoring work we're doing in this area right now.

[winder]

We're in the final stages of removing the go-algorand dependency from the develop branch. The new approach for this sort of thing would be to set your custom consensus version in algod when configuring it to support Indexer / Conduit.