Skip to content

Wrap (de)mangling namespaces in an inline namespace. #30733

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 24, 2020

Conversation

allevato
Copy link
Member

@allevato allevato commented Mar 31, 2020

Since libDemangling is included in the Swift standard library,
ODR violations can occur on platforms that allow statically
linking stdlib if Swift code is linked with other compiler
libraries that also transitively pull in libDemangling, and if
the stdlib version and compiler version do not match exactly
(even down to commit drift between releases). This lets the
runtime conditionally segregate its copies of the libDemangling
symbols from those in the compiler using an inline namespace
without affecting usage throughout source.

cc: @compnerd

# that someone might want to use, and because it does not appear in source
# code.
target_compile_definitions(swiftDemangling PUBLIC
SWIFT_INLINE_NAMESPACE=cins)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is meant to be embedded within the swift namespace so I don't think that the collision is something to worry about. We could also just use __compiler.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not attached to these specific names, but I guess I wanted to make sure that if someone did something reasonable down the line, like namespace swift { namespace compiler {, it was clear that that was distinct from this macro-hidden namespace.

I pushed an amended commit with _compiler, since __compiler would technically be a reserved word IIRC.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, _compiler should be fine. Ah, okay, I see what you mean - its merely about cognitive costs.

@@ -45,7 +45,8 @@ if(("${SWIFT_HOST_VARIANT_SDK}" STREQUAL "${SWIFT_PRIMARY_VARIANT_SDK}") AND
# interfaces.
target_compile_definitions(SwiftRuntimeLongTests
PRIVATE
swiftCore_EXPORTS)
swiftCore_EXPORTS
SWIFT_INLINE_NAMESPACE=rtins)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar for the runtime, runtime or __runtime is preferable IMO.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Used _runtime for the same reasons mentioned above. PTAL.

@allevato allevato force-pushed the rt-inline-ns branch 2 times, most recently from 9d2694e to f1f009c Compare March 31, 2020 23:35
@compnerd
Copy link
Member

compnerd commented Apr 1, 2020

This probably should be rebased.

@allevato
Copy link
Member Author

allevato commented Apr 1, 2020

This is rebased on master now that #30732 is merged, and as we discussed offline, I pushed the inline namespace down into Demangle/Mangle/Punycode and just use compiler/runtime without the underscore for the name.

@allevato allevato marked this pull request as ready for review April 1, 2020 17:44
Copy link
Member

@compnerd compnerd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to verify the reason that the runtime aspect of this change is scattered is because of the custom CMake handling in the runtime preventing properly using CMake correct?

@compnerd
Copy link
Member

compnerd commented Apr 2, 2020

@swift-ci please test

@swift-ci
Copy link
Contributor

swift-ci commented Apr 2, 2020

Build failed
Swift Test OS X Platform
Git Sha - f1f009cd7c4665f6d23c1eee1b117ea63d4b0b89

@swift-ci
Copy link
Contributor

swift-ci commented Apr 2, 2020

Build failed
Swift Test Linux Platform
Git Sha - f1f009cd7c4665f6d23c1eee1b117ea63d4b0b89

@allevato
Copy link
Member Author

allevato commented Apr 2, 2020

Just to verify the reason that the runtime aspect of this change is scattered is because of the custom CMake handling in the runtime preventing properly using CMake correct?

It's a combination of the add_swift_target_library macro splatting out the platform/arch cross product of targets and some of the other various runtime-affected targets (like the unit tests) touching the libDemangling headers in their own ways. Each one was discovered by attempting a build and then adding the define if I hit linker errors.

I imagine if there was a common "choke point" that all runtime targets depended on libDemangling through, we could reduce the cognitive overhead here, but I don't know CMake well enough to know the best way to compose that. If you have any ideas, I'd be happy to try to simplify it.

@compnerd
Copy link
Member

compnerd commented Apr 3, 2020

Hmm, Im not a fan of this, but, I think that it is safe to put that into add_swift_target_library. That is guaranteed to only be used by targets that are designed to run on the target which is what the runtime is for. I'd be okay with it being a follow up if its a big change.

@compnerd
Copy link
Member

compnerd commented Apr 3, 2020

FAILED: source/Core/CMakeFiles/lldbCore.dir/Mangled.cpp.obj 
T:\Library\Developer\Toolchains\unknown-Asserts-development.xctoolchain\usr\bin\clang-cl.exe  /nologo -TP -DGTEST_HAS_RTTI=0 -DLLDB_CONFIGURATION_RELEASE -DUNICODE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -D_ENABLE_EXTENDED_ALIGNED_STORAGE -D_HAS_EXCEPTIONS=0 -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS -D_UNICODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Isource\Core -IS:\jenkins\workspace\swift-PR-windows\lldb\source\Core -Isource -IS:\jenkins\workspace\swift-PR-windows\lldb\include -Iinclude -IS:\jenkins\workspace\swift-PR-windows\llvm\include -IT:\llvm\include -IS:\jenkins\workspace\swift-PR-windows\clang\include -IT:\llvm\tools\clang\include -IT:\swift\include -IS:\jenkins\workspace\swift-PR-windows\swift\include -IS:\jenkins\workspace\swift-PR-windows\lldb\source -I"C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\include" -IS:\jenkins\workspace\swift-PR-windows\lldb\tools\clang\include -I..\clang\include -IS:\jenkins\workspace\swift-PR-windows\lldb\source\. /GS- /Oy /Zc:inline /Zc:strictStrings /Oi /Zc:rvalueCast -Wno-deprecated-declarations -Wno-unknown-pragmas -Wno-strict-aliasing -Wno-deprecated-register -Wno-vla-extension /MD /O2 /Ob2   -UNDEBUG -wd4018 -wd4068 -wd4150 -wd4201 -wd4251 -wd4521 -wd4530  /EHs-c- /GR- -std:c++14 /showIncludes /Fosource\Core\CMakeFiles\lldbCore.dir\Mangled.cpp.obj /Fdsource\Core\CMakeFiles\lldbCore.dir\lldbCore.pdb -c S:\jenkins\workspace\swift-PR-windows\lldb\source\Core\Mangled.cpp
In file included from S:\jenkins\workspace\swift-PR-windows\lldb\source\Core\Mangled.cpp:28:
In file included from S:\jenkins\workspace\swift-PR-windows\lldb\include\lldb/Target/SwiftLanguageRuntime.h:23:
In file included from S:\jenkins\workspace\swift-PR-windows\lldb\source\Plugins/LanguageRuntime/ObjC/AppleObjCRuntime/AppleObjCRuntimeV2.h:16:
In file included from S:\jenkins\workspace\swift-PR-windows\lldb\source\Plugins/LanguageRuntime/ObjC/AppleObjCRuntime/AppleObjCRuntime.h:15:
In file included from S:\jenkins\workspace\swift-PR-windows\lldb\source\Plugins/LanguageRuntime/ObjC/AppleObjCRuntime/AppleThreadPlanStepThroughObjCTrampoline.h:14:
In file included from S:\jenkins\workspace\swift-PR-windows\lldb\include\lldb/Target/ThreadPlan.h:17:
In file included from S:\jenkins\workspace\swift-PR-windows\lldb\include\lldb/Target/Target.h:33:
S:\jenkins\workspace\swift-PR-windows\lldb\include\lldb/Symbol/SwiftASTContext.h(659,16): warning: 'CreateTupleType' overrides a member function but is not marked 'override' [-Winconsistent-missing-override]
  CompilerType CreateTupleType(const std::vector<TupleElement> &elements);
               ^
S:\jenkins\workspace\swift-PR-windows\lldb\include\lldb/Symbol/SwiftASTContext.h(122,3): note: overridden virtual function is here
  CreateTupleType(const std::vector<TupleElement> &elements) = 0;
  ^
S:\jenkins\workspace\swift-PR-windows\lldb\include\lldb/Symbol/SwiftASTContext.h(661,16): warning: 'GetErrorType' overrides a member function but is not marked 'override' [-Winconsistent-missing-override]
  CompilerType GetErrorType();
               ^
S:\jenkins\workspace\swift-PR-windows\lldb\include\lldb/Symbol/SwiftASTContext.h(111,24): note: overridden virtual function is here
  virtual CompilerType GetErrorType() = 0;
                       ^
In file included from S:\jenkins\workspace\swift-PR-windows\lldb\source\Core\Mangled.cpp:30:
In file included from S:\jenkins\workspace\swift-PR-windows\swift\include\swift/Demangling/Demangle.h:28:
S:\jenkins\workspace\swift-PR-windows\swift\include\swift/Demangling/NamespaceMacros.h(29,2): error: Libraries that depend on Demangling must set "SWIFT_INLINE_NAMESPACE"        to be a valid identifier.
#error Libraries that depend on Demangling must set "SWIFT_INLINE_NAMESPACE" \
 ^
In file included from S:\jenkins\workspace\swift-PR-windows\lldb\source\Core\Mangled.cpp:30:
S:\jenkins\workspace\swift-PR-windows\swift\include\swift/Demangling/Demangle.h(36,1): error: unknown type name 'SWIFT_BEGIN_INLINE_NAMESPACE'
SWIFT_BEGIN_INLINE_NAMESPACE
^
S:\jenkins\workspace\swift-PR-windows\swift\include\swift/Demangling/Demangle.h(620,1): error: unknown type name 'SWIFT_END_INLINE_NAMESPACE'
SWIFT_END_INLINE_NAMESPACE
^
S:\jenkins\workspace\swift-PR-windows\swift\include\swift/Demangling/Demangle.h(621,1): error: expected unqualified-id
} // end namespace Demangle
^
2 warnings and 4 errors generated.

@allevato
Copy link
Member Author

allevato commented Apr 3, 2020

It looks like lldb may be including Demangling without having the dependency declared, so it's not inheriting the compile definition. I'll verify and prep a PR for that.

For the other change, what do you think about just creating two separate swiftDemangling targets? swiftDemangling would remain as is, and we could also have swiftDemanglingRuntime that has the same sources but defines SWIFT_INLINE_NAMESPACE=runtime instead. Then as long as the targets declare the dependency on the correct target, they'll automatically get the correct definition. Do you see any problems with that approach?

@compnerd
Copy link
Member

compnerd commented Apr 3, 2020

I would say that is the correct approach ... if its not Swift :-(. The separate library should be an OBJECT library that gets subsumed by swiftCore. Sadly, this approach is currently not viable due to the way that the CMake for this project is setup. But, you are approaching the problem precisely as it should be handled. It would obviate a bunch of workarounds and difficulty in this. Perhaps Im missing something (or Im not thinking of a clever workaround) but it seems that its going to be a bit more work to get that working.

allevato added a commit to allevato/llvm-project that referenced this pull request Apr 3, 2020
swiftlang/swift#30733 will begin requiring
this in a change that allows the swiftDemangling symbols to be
conditionally renamed using an inline namespace so that compiler
and runtime symbols don't collide if statically linked in the
same binary.

The namespace `compiler` was chosen for lldb since that's what
would have been used if its build was properly inheriting the
public definition from the Swift CMake target.
allevato added a commit to allevato/llvm-project that referenced this pull request Apr 3, 2020
swiftlang/swift#30733 will begin requiring
this in a change that allows the swiftDemangling symbols to be
conditionally renamed using an inline namespace so that compiler
and runtime symbols don't collide if statically linked in the
same binary.

The namespace `compiler` was chosen for lldb since that's what
would have been used if its build was properly inheriting the
public definition from the Swift CMake target.
@allevato allevato force-pushed the rt-inline-ns branch 2 times, most recently from 7caa63f to 5f7952b Compare April 3, 2020 20:53
@allevato
Copy link
Member Author

allevato commented Apr 3, 2020

Ok, I've pushed an update that pulls the compile definition up into _add_swift_target_library_single, which cut the number of files changed from 26 to 17. I also noticed that lib/SwiftRemoteMirror was using the wrong definition (it should have had a dependency to host swiftDemangling), so I added that.

The only place where the runtime namespace is repeated now is in the unit test targets that link to target Demangling. It's possible we might be able to conditionalize that somehow, but since the same add_swift_unittest macro is used for host and target tests, it seemed better as a follow-up.

@allevato
Copy link
Member Author

@compnerd I've fixed up recent merge conflicts; now that the LLDB changes have been merged, can you kick off CI again please?

@compnerd
Copy link
Member

@swift-ci please test

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 2ab34f2d5d6de601989df0a4730a531b27ef8a29

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - 2ab34f2d5d6de601989df0a4730a531b27ef8a29

@allevato
Copy link
Member Author

allevato commented Jun 1, 2020

Now that lldb is forward-declaring symbols in swift::Demangling:: without including the header, adding the namespace for the compiler's (and thus lldb's) version of the symbols won't work cleanly. I've opened swiftlang/llvm-project#1295 to revert those changes and updated this PR to only add the inline namespace if the SWIFT_INLINE_NAMESPACE symbol is defined for the runtime. Otherwise, the macros do nothing.

@compnerd
Copy link
Member

compnerd commented Jun 2, 2020

@swift-ci please test

@swift-ci
Copy link
Contributor

swift-ci commented Jun 3, 2020

Build failed
Swift Test OS X Platform
Git Sha - f1018aee24e12bf624ce128c5d346537e01b8ff6

@compnerd
Copy link
Member

@swift-ci please clean test

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - f1018aee24e12bf624ce128c5d346537e01b8ff6

@compnerd
Copy link
Member

@swift-ci please clean test

@compnerd
Copy link
Member

@swift-ci please test Windows platform

@compnerd
Copy link
Member

@swift-ci please clean test macOS platform

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - d8ac4d154c383eb36a58694b16a1266f28d38256

@compnerd
Copy link
Member

@swift-ci please clean test

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 3c42c4126c92ec4e1e046b4c1aaa645433a1ff3f

Since libDemangling is included in the Swift standard library,
ODR violations can occur on platforms that allow statically
linking stdlib if Swift code is linked with other compiler
libraries that also transitively pull in libDemangling, and if
the stdlib version and compiler version do not match exactly
(even down to commit drift between releases). This lets the
runtime conditionally segregate its copies of the libDemangling
symbols from those in the compiler using an inline namespace
without affecting usage throughout source.
@compnerd
Copy link
Member

@swift-ci please clean test

@compnerd
Copy link
Member

@swift-ci please test Windows platform

@compnerd
Copy link
Member

CC: @mikeash @slavapestov

@compnerd compnerd merged commit 46c4c2b into swiftlang:master Jun 24, 2020
@allevato allevato deleted the rt-inline-ns branch June 24, 2020 15:35
edymtt added a commit to edymtt/swift that referenced this pull request Jan 8, 2025
This matches the behavior introduced with swiftlang#30733.

Addresses rdar://142550635
edymtt added a commit to edymtt/swift that referenced this pull request Jan 8, 2025
This matches the behavior introduced with swiftlang#30733 -- quoting the
explanation here for the sake of convenience:

> Since libDemangling is included in the Swift standard library,
> ODR violations can occur on platforms that allow statically
> linking stdlib if Swift code is linked with other compiler
> libraries that also transitively pull in libDemangling, and if
> the stdlib version and compiler version do not match exactly
> (even down to commit drift between releases). This lets the
> runtime conditionally segregate its copies of the libDemangling
> symbols from those in the compiler using an inline namespace
> without affecting usage throughout source.

Addresses rdar://142550635
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants