Skip to content

[llvm] add tool to verify mustache library #111487

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 79 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
79 commits
Select commit Hold shift + click to select a range
ac1c7b5
[clang-doc] add suport for clang-doc enum generation
PeterChou1 Jul 31, 2024
bcc4b0d
[clang-doc] remove useless code
PeterChou1 Jul 31, 2024
5fe47ca
[clang-doc] modify unittest
PeterChou1 Jul 31, 2024
28fb40f
[clang-doc] address pr comments
PeterChou1 Aug 12, 2024
0d150ea
[clang-doc] revert CommentInfo change
PeterChou1 Aug 12, 2024
e5e70b8
[clang-doc] fix test
PeterChou1 Aug 12, 2024
1c4f631
[clang-doc] fix unittest
PeterChou1 Aug 12, 2024
b8f3f9c
[clang-doc] address pr comments
PeterChou1 Aug 12, 2024
bb98aad
[clang-doc] clang-format
PeterChou1 Aug 12, 2024
c60e2b5
[llvm] implement support for mustache template language
PeterChou1 Aug 23, 2024
0a20de8
Merge branch 'main' into llvm-add-mustache
PeterChou1 Aug 23, 2024
8290c38
[llvm][Support] Finish implementation of Mustache
PeterChou1 Sep 6, 2024
07d31b4
Merge branch 'llvm-add-mustache' of https://github.com/PeterChou1/llv…
PeterChou1 Sep 6, 2024
6e893c0
[llvm][support] fix mustache test
PeterChou1 Sep 6, 2024
976593e
[llvm][support] add comments
PeterChou1 Sep 6, 2024
6a60eab
[llvm][support] add comments
PeterChou1 Sep 6, 2024
6a1fcc8
[llvm][support] use enumerate for loop
PeterChou1 Sep 6, 2024
f1c27af
Merge branch 'main' into llvm-add-mustache
PeterChou1 Sep 6, 2024
eb1e1a6
[llvm][support] clang-format
PeterChou1 Sep 6, 2024
8ff1100
Merge branch 'llvm-add-mustache' of https://github.com/PeterChou1/llv…
PeterChou1 Sep 6, 2024
f4b0520
[llvm][support] fix mustache test
PeterChou1 Sep 6, 2024
7ffaeec
[llvm][support] clang-format
PeterChou1 Sep 6, 2024
746fb97
[llvm][support] use llvm enumerate
PeterChou1 Sep 6, 2024
4944435
Merge branch 'main' into llvm-add-mustache
PeterChou1 Sep 6, 2024
bcc86fe
[llvm][support] fix unittest
PeterChou1 Sep 11, 2024
2ceb0b0
Merge branch 'llvm-add-mustache' of https://github.com/PeterChou1/llv…
PeterChou1 Sep 11, 2024
95ad7a6
[llvm][support] clang-format
PeterChou1 Sep 11, 2024
bb3b1ac
[llvm][support] address mustache comments
PeterChou1 Sep 12, 2024
d8aa85c
[llvm][support] clang-format
PeterChou1 Sep 12, 2024
0534a05
[llvm][support] clang-format
PeterChou1 Sep 12, 2024
06da7a5
[llvm][support] clang-format
PeterChou1 Sep 12, 2024
f713198
[llvm] mustache address comments
PeterChou1 Oct 8, 2024
6b4f5cd
[llvm] clang-format
PeterChou1 Oct 8, 2024
d400c29
[llvm] fix errors
PeterChou1 Oct 8, 2024
8fa5fd7
[llvm] fix more bugs
PeterChou1 Oct 8, 2024
fd7c106
[llvm] change mustache to pass by reference
PeterChou1 Oct 8, 2024
29bba68
[llvm] fix failing mustache regression test
PeterChou1 Oct 8, 2024
7eed82f
[llvm] format
PeterChou1 Oct 8, 2024
e73454d
[llvm] remove unused mustache member
PeterChou1 Oct 8, 2024
b58fbcb
[llvm] remove unused mustache member
PeterChou1 Oct 8, 2024
bb4ba40
[llvm] address more comments
PeterChou1 Oct 8, 2024
b0ce196
[llvm] address comments
PeterChou1 Oct 8, 2024
561c7eb
clang-format
PeterChou1 Oct 8, 2024
96f6990
clang-format
PeterChou1 Oct 8, 2024
a247423
[llvm] factor out internal classes
PeterChou1 Oct 10, 2024
a0e7d48
[llvm] clang-format
PeterChou1 Oct 10, 2024
4165fec
[llvm] refactor mustache to raw_ostream
PeterChou1 Oct 12, 2024
5b0a20a
[llvm] refactor to use os_stream
PeterChou1 Oct 12, 2024
ba4a6db
[llvm] clang-format
PeterChou1 Oct 12, 2024
362728f
[llvm] fix indentation bug
PeterChou1 Oct 12, 2024
dca40c5
[llvm] add mustache verification tool
PeterChou1 Oct 8, 2024
6fffeb2
[llvm] modify mustache tool
PeterChou1 Oct 12, 2024
7337a99
[llvm] add mustache verification tool
PeterChou1 Oct 8, 2024
439955b
[llvm] add mustache verification tool
PeterChou1 Oct 8, 2024
c7ba28b
Merge branch 'main' into llvm-add-mustache-verify-tool
PeterChou1 Mar 26, 2025
756c3a3
address comments
PeterChou1 Apr 2, 2025
db52154
change filename
PeterChou1 Apr 2, 2025
1a23334
fix comments
PeterChou1 Apr 2, 2025
2e01630
Merge branch 'llvm-add-mustache-verify-tool' of https://github.com/Pe…
PeterChou1 Apr 2, 2025
69e5fbb
address more comments
PeterChou1 Apr 2, 2025
d812837
format
PeterChou1 Apr 2, 2025
d576d69
format
PeterChou1 Apr 2, 2025
6985fcc
format
PeterChou1 Apr 2, 2025
cfa8e5e
add documentation
PeterChou1 Apr 2, 2025
fe6d4ba
format
PeterChou1 Apr 2, 2025
093b732
fix doc
PeterChou1 Apr 2, 2025
6ff0b7e
add doc to toctree
PeterChou1 Apr 2, 2025
fa558e5
move to utils
PeterChou1 Apr 3, 2025
dbfdb5d
fix docs
PeterChou1 Apr 3, 2025
35d717e
move doc
PeterChou1 Apr 3, 2025
c85fb69
fix doc errors
PeterChou1 Apr 3, 2025
c238bbc
rename
PeterChou1 Apr 3, 2025
6bf2231
fix index
PeterChou1 Apr 3, 2025
a439e55
change cmake
PeterChou1 Apr 3, 2025
6a73432
fix doc
PeterChou1 Apr 3, 2025
ee21301
fix doc
PeterChou1 Apr 3, 2025
8228800
fix doc
PeterChou1 Apr 3, 2025
f6aa0c5
fix doc
PeterChou1 Apr 3, 2025
f7a68bd
add description
PeterChou1 Apr 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions llvm/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -1276,6 +1276,7 @@ if( LLVM_INCLUDE_UTILS )
add_subdirectory(utils/yaml-bench)
add_subdirectory(utils/split-file)
add_subdirectory(utils/mlgo-utils)
add_subdirectory(utils/llvm-mustachespec)
if( LLVM_INCLUDE_TESTS )
set(LLVM_SUBPROJECT_TITLE "Third-Party/Google Test")
add_subdirectory(${LLVM_THIRD_PARTY_DIR}/unittest ${CMAKE_CURRENT_BINARY_DIR}/third-party/unittest)
Expand Down
1 change: 1 addition & 0 deletions llvm/docs/CommandGuide/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,7 @@ Developer Tools
llvm-pdbutil
llvm-profgen
llvm-tli-checker
llvm-mustachespec

Remarks Tools
~~~~~~~~~~~~~~
Expand Down
12 changes: 12 additions & 0 deletions llvm/docs/CommandGuide/llvm-mustachespec.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
llvm-mustachespec - LLVM tool to test Mustache Compliance Library
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
llvm-mustachespec - LLVM tool to test Mustache Compliance Library
llvm-mustachespec - LLVM tool to test Mustache Library Compliance

Possibly?

=================================================================

llvm-mustachespec test the mustache spec conformance of the LLVM
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
llvm-mustachespec test the mustache spec conformance of the LLVM
llvm-mustachespec tests the mustache spec conformance of the LLVM

mustache library. The spec can be found here https://github.com/mustache/spec
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
mustache library. The spec can be found here https://github.com/mustache/spec
mustache library. The spec can be found here: `Mustache Spec <https://github.com/mustache/spec>`_.

or something.


$ llvm-mustachespec input-file

.. program:: llvm-mustachespec

Outputs the number of tests failures and success in the spec
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Outputs the number of tests failures and success in the spec
Outputs the number of tests failures and successes in the spec.


5 changes: 5 additions & 0 deletions llvm/utils/llvm-mustachespec/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
add_llvm_utility(llvm-mustachespec
llvm-mustachespec.cpp
)

target_link_libraries(llvm-mustachespec PRIVATE LLVMSupport)
104 changes: 104 additions & 0 deletions llvm/utils/llvm-mustachespec/llvm-mustachespec.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
//===- llvm-mustachespec.cpp - The LLVM Modular Optimizer
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
//===- llvm-mustachespec.cpp - The LLVM Modular Optimizer

We no longer include this line.

//===----------------------------------------------------------------------===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
//
// Simple drivers to test the mustache spec found here
// https://github.com/mustache/
// It is used to verify that the current implementation conforms to the spec
// simply download the spec and pass the test files to the driver
Comment on lines +10 to +13
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Simple drivers to test the mustache spec found here
// https://github.com/mustache/
// It is used to verify that the current implementation conforms to the spec
// simply download the spec and pass the test files to the driver
// Simple drivers to test the mustache spec found here:
// https://github.com/mustache/
// It is used to verify that the current implementation conforms to the spec.
// Simply download the spec and pass the test files to the driver.

//
// Currently Triple Mustache is not supported we expect the following spec
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Currently Triple Mustache is not supported we expect the following spec
// Currently Triple Mustache is not supported, so we expect the following spec

// test to fail:
// Triple Mustache
// Triple Mustache Integer Interpolation
// Triple Mustache Decimal Interpolation
// Triple Mustache Null Interpolation
// Triple Mustache Context Miss Interpolation
// Dotted Names - Triple Mustache Interpolation
// Implicit Iterators - Triple Mustache
// Triple Mustache - Surrounding Whitespace
// Triple Mustache - Standalone
// Triple Mustache With Padding
// Standalone Indentation
// Implicit Iterator - Triple mustache
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder whether it would make sense to include this list directly in the program, and report these as "xfail"? That way it's easy to see whether there are any unexpected failures without manually comparing to this list.

//
// Usage:
// mustache path/to/test/file/test.json path/to/test/file/test2.json ...
//===----------------------------------------------------------------------===//

#include "llvm/Support/CommandLine.h"
#include "llvm/Support/MemoryBuffer.h"
#include "llvm/Support/Mustache.h"
#include <string>

using namespace llvm;
using namespace llvm::json;
using namespace llvm::mustache;

cl::list<std::string> InputFiles(cl::Positional, cl::desc("<input files>"),
cl::OneOrMore);

void runThroughTest(StringRef InputFile) {
llvm::outs() << "Running Tests: " << InputFile << "\n";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
llvm::outs() << "Running Tests: " << InputFile << "\n";
outs() << "Running Tests: " << InputFile << "\n";

You're using using namespace llvm, don't need all these llvm:: prefixes.

ErrorOr<std::unique_ptr<llvm::MemoryBuffer>> BufferOrError =
MemoryBuffer::getFile(InputFile);

if (auto EC = BufferOrError.getError()) {
return;
}
Comment on lines +51 to +53
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't the error get reported?

std::unique_ptr<llvm::MemoryBuffer> Buffer = std::move(BufferOrError.get());
llvm::StringRef FileContent = Buffer->getBuffer();
Expected<Value> Json = parse(FileContent);

if (auto E = Json.takeError()) {
errs() << "Parsing error: " << toString(std::move(E)) << "\n";
return;
}
// Get test
Array *Obj = (*Json).getAsObject()->getArray("tests");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Array *Obj = (*Json).getAsObject()->getArray("tests");
Array *Obj = Json->getAsObject()->getArray("tests");

This looks a bit odd, does this work?

size_t Total = 0;
size_t Success = 0;
for (Value V : *Obj) {
Object *TestCase = V.getAsObject();
StringRef TemplateStr = TestCase->getString("template").value();
StringRef ExpectedStr = TestCase->getString("expected").value();
StringRef Name = TestCase->getString("name").value();
Value *Data = TestCase->get("data");
Value *Partials = TestCase->get("partials");

if (!Data)
continue;

Template T = Template(TemplateStr);
if (Partials) {
for (auto &PartialPairs : *Partials->getAsObject()) {
const auto &[Partial, Str] = PartialPairs;
T.registerPartial((*Str.getAsString()).str(), Partial.str());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
T.registerPartial((*Str.getAsString()).str(), Partial.str());
T.registerPartial(Str.getAsString()->str(), Partial.str());

}
}
std::string ActualStr;
llvm::raw_string_ostream OS(ActualStr);
T.render(*Data, OS);
if (ExpectedStr == ActualStr) {
Success++;
} else {
llvm::outs() << "Test Failed: " << Name << "\n";
}
Total++;
}

llvm::outs() << "Result " << Success << "/" << Total << " succeeded\n";
}

int main(int argc, char **argv) {
llvm::cl::ParseCommandLineOptions(argc, argv);
for (const auto &FileName : InputFiles) {
runThroughTest(FileName);
}
return 0;
}