[LoongArch] Support sc.q instruction for 128bit cmpxchg operation #116771

tangaac · 2024-11-19T09:39:15Z

Two options for clang
-mno-scq: Disable sc.q instruction.
-mscq: Enable sc.q instruction.
The default is -mno-scq.

llvmbot · 2024-11-19T09:39:52Z

@llvm/pr-subscribers-backend-loongarch
@llvm/pr-subscribers-clang-driver

@llvm/pr-subscribers-clang

Author: None (tangaac)

Changes

Two options for clang
-mno-scq: Disable sc.q instruction.
-mscq: Enable sc.q instruction.
The default is -mno-scq.

Patch is 36.49 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/116771.diff

15 Files Affected:

(modified) clang/include/clang/Driver/Options.td (+4)
(modified) clang/lib/Basic/Targets/LoongArch.cpp (+6-1)
(modified) clang/lib/Basic/Targets/LoongArch.h (+2)
(modified) clang/lib/Driver/ToolChains/Arch/LoongArch.cpp (+9)
(modified) clang/test/Driver/loongarch-march.c (+4-4)
(added) clang/test/Driver/loongarch-mscq.c (+30)
(modified) clang/test/Preprocessor/init-loongarch.c (+17-8)
(modified) llvm/include/llvm/TargetParser/LoongArchTargetParser.def (+2-1)
(modified) llvm/include/llvm/TargetParser/LoongArchTargetParser.h (+3)
(modified) llvm/lib/Target/LoongArch/LoongArch.td (+8-1)
(modified) llvm/lib/Target/LoongArch/LoongArchExpandAtomicPseudoInsts.cpp (+107)
(modified) llvm/lib/Target/LoongArch/LoongArchISelLowering.cpp (+46)
(modified) llvm/lib/Target/LoongArch/LoongArchInstrInfo.td (+14)
(modified) llvm/lib/TargetParser/LoongArchTargetParser.cpp (+1)
(added) llvm/test/CodeGen/LoongArch/ir-instruction/atomic-cmpxchg-128.ll (+287)

diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index f2f9c20c9bc264..3362bed1bea0a3 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -5417,6 +5417,10 @@ def mlam_bh : Flag<["-"], "mlam-bh">, Group<m_loongarch_Features_Group>,
   HelpText<"Enable amswap[_db].{b/h} and amadd[_db].{b/h}">;
 def mno_lam_bh : Flag<["-"], "mno-lam-bh">, Group<m_loongarch_Features_Group>,
   HelpText<"Disable amswap[_db].{b/h} and amadd[_db].{b/h}">;
+def mscq : Flag<["-"], "mscq">, Group<m_loongarch_Features_Group>,
+  HelpText<"Enable sc.q instruction.">;
+def mno_scq : Flag<["-"], "mno-scq">, Group<m_loongarch_Features_Group>,
+  HelpText<"Disable sc.q instruction.">;
 def mannotate_tablejump : Flag<["-"], "mannotate-tablejump">, Group<m_loongarch_Features_Group>,
   HelpText<"Enable annotate table jump instruction to correlate it with the jump table.">;
 def mno_annotate_tablejump : Flag<["-"], "mno-annotate-tablejump">, Group<m_loongarch_Features_Group>,
diff --git a/clang/lib/Basic/Targets/LoongArch.cpp b/clang/lib/Basic/Targets/LoongArch.cpp
index 07b22b35f603ce..088da3272e0f85 100644
--- a/clang/lib/Basic/Targets/LoongArch.cpp
+++ b/clang/lib/Basic/Targets/LoongArch.cpp
@@ -205,7 +205,7 @@ void LoongArchTargetInfo::getTargetDefines(const LangOptions &Opts,
       // TODO: As more features of the V1.1 ISA are supported, a unified "v1.1"
       // arch feature set will be used to include all sub-features belonging to
       // the V1.1 ISA version.
-      if (HasFeatureFrecipe && HasFeatureLAM_BH)
+      if (HasFeatureFrecipe && HasFeatureLAM_BH && HasFeatureSCQ)
         Builder.defineMacro("__loongarch_arch",
                             Twine('"') + "la64v1.1" + Twine('"'));
       else
@@ -239,6 +239,9 @@ void LoongArchTargetInfo::getTargetDefines(const LangOptions &Opts,
   if (HasFeatureLAM_BH)
     Builder.defineMacro("__loongarch_lam_bh", Twine(1));
 
+  if (HasFeatureSCQ)
+    Builder.defineMacro("__loongarch_scq", Twine(1));
+
   StringRef ABI = getABI();
   if (ABI == "lp64d" || ABI == "lp64f" || ABI == "lp64s")
     Builder.defineMacro("__loongarch_lp64");
@@ -317,6 +320,8 @@ bool LoongArchTargetInfo::handleTargetFeatures(
       HasFeatureFrecipe = true;
     else if (Feature == "+lam-bh")
       HasFeatureLAM_BH = true;
+    else if (Feature == "+scq")
+      HasFeatureSCQ = true;
   }
   return true;
 }
diff --git a/clang/lib/Basic/Targets/LoongArch.h b/clang/lib/Basic/Targets/LoongArch.h
index 3585e9f7968b4b..cd8cea0b54e045 100644
--- a/clang/lib/Basic/Targets/LoongArch.h
+++ b/clang/lib/Basic/Targets/LoongArch.h
@@ -31,6 +31,7 @@ class LLVM_LIBRARY_VISIBILITY LoongArchTargetInfo : public TargetInfo {
   bool HasFeatureLASX;
   bool HasFeatureFrecipe;
   bool HasFeatureLAM_BH;
+  bool HasFeatureSCQ;
 
 public:
   LoongArchTargetInfo(const llvm::Triple &Triple, const TargetOptions &)
@@ -41,6 +42,7 @@ class LLVM_LIBRARY_VISIBILITY LoongArchTargetInfo : public TargetInfo {
     HasFeatureLASX = false;
     HasFeatureFrecipe = false;
     HasFeatureLAM_BH = false;
+    HasFeatureSCQ = false;
     LongDoubleWidth = 128;
     LongDoubleAlign = 128;
     LongDoubleFormat = &llvm::APFloat::IEEEquad();
diff --git a/clang/lib/Driver/ToolChains/Arch/LoongArch.cpp b/clang/lib/Driver/ToolChains/Arch/LoongArch.cpp
index 987db4638fca88..e0454edaabb0fc 100644
--- a/clang/lib/Driver/ToolChains/Arch/LoongArch.cpp
+++ b/clang/lib/Driver/ToolChains/Arch/LoongArch.cpp
@@ -274,6 +274,15 @@ void loongarch::getLoongArchTargetFeatures(const Driver &D,
     else
       Features.push_back("-lam-bh");
   }
+
+  // Select scq feature determined by -m[no-]scq.
+  if (const Arg *A =
+          Args.getLastArg(options::OPT_mscq, options::OPT_mno_scq)) {
+    if (A->getOption().matches(options::OPT_mscq))
+      Features.push_back("+scq");
+    else
+      Features.push_back("-scq");
+  }
 }
 
 std::string loongarch::postProcessTargetCPUString(const std::string &CPU,
diff --git a/clang/test/Driver/loongarch-march.c b/clang/test/Driver/loongarch-march.c
index d4cd5b07ae905f..51e2938e447ef3 100644
--- a/clang/test/Driver/loongarch-march.c
+++ b/clang/test/Driver/loongarch-march.c
@@ -39,21 +39,21 @@
 
 // CC1-LA64V1P1: "-target-cpu" "loongarch64"
 // CC1-LA64V1P1-NOT: "-target-feature"
-// CC1-LA64V1P1: "-target-feature" "+64bit" "-target-feature" "+d" "-target-feature" "+lsx" "-target-feature" "+ual" "-target-feature" "+frecipe" "-target-feature" "+lam-bh"
+// CC1-LA64V1P1: "-target-feature" "+64bit" "-target-feature" "+d" "-target-feature" "+lsx" "-target-feature" "+ual" "-target-feature" "+frecipe" "-target-feature" "+lam-bh" "-target-feature" "+scq"
 // CC1-LA64V1P1-NOT: "-target-feature"
 // CC1-LA64V1P1: "-target-abi" "lp64d"
 
 // CC1-LA664: "-target-cpu" "la664"
 // CC1-LA664-NOT: "-target-feature"
-// CC1-LA664: "-target-feature" "+64bit" "-target-feature" "+f" "-target-feature" "+d" "-target-feature" "+lsx" "-target-feature" "+lasx" "-target-feature" "+ual" "-target-feature" "+frecipe" "-target-feature" "+lam-bh"
+// CC1-LA664: "-target-feature" "+64bit" "-target-feature" "+f" "-target-feature" "+d" "-target-feature" "+lsx" "-target-feature" "+lasx" "-target-feature" "+ual" "-target-feature" "+frecipe" "-target-feature" "+lam-bh" "-target-feature" "+scq"
 // CC1-LA664-NOT: "-target-feature"
 // CC1-LA664: "-target-abi" "lp64d"
 
 // IR-LOONGARCH64: attributes #[[#]] ={{.*}}"target-cpu"="loongarch64" {{.*}}"target-features"="+64bit,+d,+f,+ual"
 // IR-LA464: attributes #[[#]] ={{.*}}"target-cpu"="la464" {{.*}}"target-features"="+64bit,+d,+f,+lasx,+lsx,+ual"
 // IR-LA64V1P0: attributes #[[#]] ={{.*}}"target-cpu"="loongarch64" {{.*}}"target-features"="+64bit,+d,+lsx,+ual"
-// IR-LA64V1P1: attributes #[[#]] ={{.*}}"target-cpu"="loongarch64" {{.*}}"target-features"="+64bit,+d,+frecipe,+lam-bh,+lsx,+ual"
-// IR-LA664: attributes #[[#]] ={{.*}}"target-cpu"="la664" {{.*}}"target-features"="+64bit,+d,+f,+frecipe,+lam-bh,+lasx,+lsx,+ual"
+// IR-LA64V1P1: attributes #[[#]] ={{.*}}"target-cpu"="loongarch64" {{.*}}"target-features"="+64bit,+d,+frecipe,+lam-bh,+lsx,+scq,+ual"
+// IR-LA664: attributes #[[#]] ={{.*}}"target-cpu"="la664" {{.*}}"target-features"="+64bit,+d,+f,+frecipe,+lam-bh,+lasx,+lsx,+scq,+ual"
 
 int foo(void) {
   return 3;
diff --git a/clang/test/Driver/loongarch-mscq.c b/clang/test/Driver/loongarch-mscq.c
new file mode 100644
index 00000000000000..cd798ba5d8ff2b
--- /dev/null
+++ b/clang/test/Driver/loongarch-mscq.c
@@ -0,0 +1,30 @@
+/// Test -m[no]scq options.
+
+// RUN: %clang --target=loongarch64 -mscq -fsyntax-only %s -### 2>&1 | \
+// RUN:     FileCheck %s --check-prefix=CC1-SCQ
+// RUN: %clang --target=loongarch64 -mno-scq -fsyntax-only %s -### 2>&1 | \
+// RUN:     FileCheck %s --check-prefix=CC1-NO-SCQ
+// RUN: %clang --target=loongarch64 -mno-scq -mscq -fsyntax-only %s -### 2>&1 | \
+// RUN:     FileCheck %s --check-prefix=CC1-SCQ
+// RUN: %clang --target=loongarch64  -mscq -mno-scq -fsyntax-only %s -### 2>&1 | \
+// RUN:     FileCheck %s --check-prefix=CC1-NO-SCQ
+
+// RUN: %clang --target=loongarch64 -mscq -S -emit-llvm %s -o - | \
+// RUN: FileCheck %s --check-prefix=IR-SCQ
+// RUN: %clang --target=loongarch64 -mno-scq -S -emit-llvm %s -o - | \
+// RUN: FileCheck %s --check-prefix=IR-NO-SCQ
+// RUN: %clang --target=loongarch64 -mno-scq -mscq -S -emit-llvm %s -o - | \
+// RUN: FileCheck %s --check-prefix=IR-SCQ
+// RUN: %clang --target=loongarch64 -mscq -mno-scq -S -emit-llvm %s -o - | \
+// RUN: FileCheck %s --check-prefix=IR-NO-SCQ
+
+
+// CC1-SCQ: "-target-feature" "+scq"
+// CC1-NO-SCQ: "-target-feature" "-scq"
+
+// IR-SCQ: attributes #[[#]] ={{.*}}"target-features"="{{(.*,)?}}+scq{{(,.*)?}}"
+// IR-NO-SCQ: attributes #[[#]] ={{.*}}"target-features"="{{(.*,)?}}-scq{{(,.*)?}}"
+
+int foo(void) {
+  return 42;
+}
\ No newline at end of file
diff --git a/clang/test/Preprocessor/init-loongarch.c b/clang/test/Preprocessor/init-loongarch.c
index 8019292e0f10e0..7ae708b6ed761c 100644
--- a/clang/test/Preprocessor/init-loongarch.c
+++ b/clang/test/Preprocessor/init-loongarch.c
@@ -798,7 +798,7 @@
 // LA64-FPU0-LP64S-NOT: #define __loongarch_single_float
 // LA64-FPU0-LP64S: #define __loongarch_soft_float 1
 
-/// Check __loongarch_arch{_tune/_frecipe/_lam_bh}.
+/// Check __loongarch_arch{_tune/_frecipe/_lam_bh/_scq}.
 
 // RUN: %clang --target=loongarch64 -x c -E -dM %s -o - | \
 // RUN:   FileCheck --match-full-lines --check-prefix=ARCH-TUNE -DARCH=la64v1.0 -DTUNE=loongarch64 %s
@@ -823,11 +823,11 @@
 // RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=loongarch64 -Xclang -target-feature -Xclang +lsx | \
 // RUN:   FileCheck --match-full-lines --check-prefix=ARCH-TUNE -DARCH=la64v1.0 -DTUNE=loongarch64 %s
 // RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=la64v1.1 | \
-// RUN:   FileCheck --match-full-lines  --check-prefixes=ARCH-TUNE,FRECIPE,LAM-BH -DARCH=la64v1.1 -DTUNE=loongarch64 %s
+// RUN:   FileCheck --match-full-lines  --check-prefixes=ARCH-TUNE,FRECIPE,LAM-BH,SCQ -DARCH=la64v1.1 -DTUNE=loongarch64 %s
 // RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=la64v1.1 -Xclang -target-feature -Xclang -frecipe | \
-// RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,LAM-BH -DARCH=la64v1.0 -DTUNE=loongarch64 %s
+// RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,LAM-BH,SCQ -DARCH=la64v1.0 -DTUNE=loongarch64 %s
 // RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=la64v1.1 -Xclang -target-feature -Xclang -lsx | \
-// RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,FRECIPE,LAM-BH -DARCH=loongarch64 -DTUNE=loongarch64 %s
+// RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,FRECIPE,LAM-BH,SCQ -DARCH=loongarch64 -DTUNE=loongarch64 %s
 // RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=loongarch64 -Xclang -target-feature -Xclang +frecipe | \
 // RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,FRECIPE -DARCH=loongarch64 -DTUNE=loongarch64 %s
 // RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=loongarch64 -Xclang -target-feature -Xclang +lsx -Xclang -target-feature -Xclang +frecipe | \
@@ -835,25 +835,34 @@
 // RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=la64v1.0 -Xclang -target-feature -Xclang +lam-bh | \
 // RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,LAM-BH -DARCH=la64v1.0 -DTUNE=loongarch64 %s
 // RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=la64v1.1 -Xclang -target-feature -Xclang -lam-bh | \
-// RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,FRECIPE -DARCH=la64v1.0 -DTUNE=loongarch64 %s
+// RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,FRECIPE,SCQ -DARCH=la64v1.0 -DTUNE=loongarch64 %s
 // RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=loongarch64 -Xclang -target-feature -Xclang +lam-bh | \
 // RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,LAM-BH -DARCH=loongarch64 -DTUNE=loongarch64 %s
 // RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=loongarch64 -Xclang -target-feature -Xclang +lsx -Xclang -target-feature -Xclang +lam-bh | \
 // RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,LAM-BH -DARCH=la64v1.0 -DTUNE=loongarch64 %s
-// RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=la64v1.0 -Xclang -target-feature -Xclang +frecipe -Xclang -target-feature -Xclang +lam-bh | \
+// RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=la64v1.0 -Xclang -target-feature -Xclang +scq | \
+// RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,SCQ -DARCH=la64v1.0 -DTUNE=loongarch64 %s
+// RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=la64v1.1 -Xclang -target-feature -Xclang -scq | \
+// RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,FRECIPE,LAM-BH -DARCH=la64v1.0 -DTUNE=loongarch64 %s
+// RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=loongarch64 -Xclang -target-feature -Xclang +scq | \
+// RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,SCQ -DARCH=loongarch64 -DTUNE=loongarch64 %s
+// RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=loongarch64 -Xclang -target-feature -Xclang +lsx -Xclang -target-feature -Xclang +scq | \
+// RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,SCQ -DARCH=la64v1.0 -DTUNE=loongarch64 %s
+// RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=la64v1.0 -Xclang -target-feature -Xclang +frecipe -Xclang -target-feature -Xclang +lam-bh -Xclang -target-feature -Xclang +scq | \
 // RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE -DARCH=la64v1.1 -DTUNE=loongarch64 %s
 // RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=la664 | \
-// RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,FRECIPE,LAM-BH -DARCH=la664 -DTUNE=la664 %s
+// RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,FRECIPE,LAM-BH,SCQ -DARCH=la664 -DTUNE=la664 %s
 // RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -mtune=la664 | \
 // RUN:   FileCheck --match-full-lines --check-prefix=ARCH-TUNE -DARCH=la64v1.0 -DTUNE=la664 %s
 // RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=loongarch64 -mtune=la664 | \
 // RUN:   FileCheck --match-full-lines --check-prefix=ARCH-TUNE -DARCH=loongarch64 -DTUNE=la664 %s
 // RUN: %clang --target=loongarch64 -x c -E -dM %s -o - -march=la664 -mtune=loongarch64 | \
-// RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,FRECIPE,LAM-BH -DARCH=la664 -DTUNE=loongarch64 %s
+// RUN:   FileCheck --match-full-lines --check-prefixes=ARCH-TUNE,FRECIPE,LAM-BH,SCQ -DARCH=la664 -DTUNE=loongarch64 %s
 
 // ARCH-TUNE: #define __loongarch_arch "[[ARCH]]"
 // FRECIPE: #define __loongarch_frecipe 1
 // LAM-BH: #define __loongarch_lam_bh 1
+// SCQ: #define __loongarch_scq 1
 // ARCH-TUNE: #define __loongarch_tune "[[TUNE]]"
 
 // RUN: %clang --target=loongarch64 -mlsx -x c -E -dM %s -o - \
diff --git a/llvm/include/llvm/TargetParser/LoongArchTargetParser.def b/llvm/include/llvm/TargetParser/LoongArchTargetParser.def
index 6cd2018b7b59cb..0a8768047a9702 100644
--- a/llvm/include/llvm/TargetParser/LoongArchTargetParser.def
+++ b/llvm/include/llvm/TargetParser/LoongArchTargetParser.def
@@ -12,6 +12,7 @@ LOONGARCH_FEATURE("+lvz", FK_LVZ)
 LOONGARCH_FEATURE("+ual", FK_UAL)
 LOONGARCH_FEATURE("+frecipe", FK_FRECIPE)
 LOONGARCH_FEATURE("+lam-bh", FK_LAM_BH)
+LOONGARCH_FEATURE("+scq", FK_SCQ)
 
 #undef LOONGARCH_FEATURE
 
@@ -21,6 +22,6 @@ LOONGARCH_FEATURE("+lam-bh", FK_LAM_BH)
 
 LOONGARCH_ARCH("loongarch64", AK_LOONGARCH64, FK_64BIT | FK_FP32 | FK_FP64 | FK_UAL)
 LOONGARCH_ARCH("la464", AK_LA464, FK_64BIT | FK_FP32 | FK_FP64 | FK_LSX | FK_LASX | FK_UAL)
-LOONGARCH_ARCH("la664", AK_LA664, FK_64BIT | FK_FP32 | FK_FP64 | FK_LSX | FK_LASX | FK_UAL | FK_FRECIPE | FK_LAM_BH)
+LOONGARCH_ARCH("la664", AK_LA664, FK_64BIT | FK_FP32 | FK_FP64 | FK_LSX | FK_LASX | FK_UAL | FK_FRECIPE | FK_LAM_BH | FK_SCQ)
 
 #undef LOONGARCH_ARCH
diff --git a/llvm/include/llvm/TargetParser/LoongArchTargetParser.h b/llvm/include/llvm/TargetParser/LoongArchTargetParser.h
index b5be03b1b67fbb..2aaa0e30593793 100644
--- a/llvm/include/llvm/TargetParser/LoongArchTargetParser.h
+++ b/llvm/include/llvm/TargetParser/LoongArchTargetParser.h
@@ -53,6 +53,9 @@ enum FeatureKind : uint32_t {
   // Atomic memory swap and add instructions for byte and half word are
   // available.
   FK_LAM_BH = 1 << 10,
+
+  // sc.q is available.
+  FK_SCQ = 1 << 14,
 };
 
 struct FeatureInfo {
diff --git a/llvm/lib/Target/LoongArch/LoongArch.td b/llvm/lib/Target/LoongArch/LoongArch.td
index ecd00cd6d5d619..6d6379fc82cbbe 100644
--- a/llvm/lib/Target/LoongArch/LoongArch.td
+++ b/llvm/lib/Target/LoongArch/LoongArch.td
@@ -118,6 +118,12 @@ def FeatureLAM_BH
                         "Support amswap[_db].{b/h} and amadd[_db].{b/h} instructions.">;
 def HasLAM_BH : Predicate<"Subtarget->hasLAM_BH()">;
 
+// Support SC.Q instruction
+def FeatureSCQ
+    : SubtargetFeature<"scq", "HasSCQ", "true",
+                        "Support sc.q instruction">;
+def HasSCQ : Predicate<"Subtarget->hasSCQ()">;
+
 def TunePreferWInst
     : SubtargetFeature<"prefer-w-inst", "PreferWInst", "true",
                        "Prefer instructions with W suffix">;
@@ -160,7 +166,8 @@ def : ProcessorModel<"la664", NoSchedModel, [Feature64Bit,
                                              FeatureExtLVZ,
                                              FeatureExtLBT,
                                              FeatureFrecipe,
-                                             FeatureLAM_BH]>;
+                                             FeatureLAM_BH,
+                                             FeatureSCQ]>;
 
 //===----------------------------------------------------------------------===//
 // Define the LoongArch target.
diff --git a/llvm/lib/Target/LoongArch/LoongArchExpandAtomicPseudoInsts.cpp b/llvm/lib/Target/LoongArch/LoongArchExpandAtomicPseudoInsts.cpp
index 18a532b55ee5a9..a3c95bbf5a8dd6 100644
--- a/llvm/lib/Target/LoongArch/LoongArchExpandAtomicPseudoInsts.cpp
+++ b/llvm/lib/Target/LoongArch/LoongArchExpandAtomicPseudoInsts.cpp
@@ -58,6 +58,9 @@ class LoongArchExpandAtomicPseudo : public MachineFunctionPass {
   bool expandAtomicCmpXchg(MachineBasicBlock &MBB,
                            MachineBasicBlock::iterator MBBI, bool IsMasked,
                            int Width, MachineBasicBlock::iterator &NextMBBI);
+  bool expandAtomicCmpXchg128(MachineBasicBlock &MBB,
+                              MachineBasicBlock::iterator,
+                              MachineBasicBlock::iterator &NextMBBI);
 };
 
 char LoongArchExpandAtomicPseudo::ID = 0;
@@ -131,6 +134,9 @@ bool LoongArchExpandAtomicPseudo::expandMI(
     return expandAtomicCmpXchg(MBB, MBBI, false, 32, NextMBBI);
   case LoongArch::PseudoCmpXchg64:
     return expandAtomicCmpXchg(MBB, MBBI, false, 64, NextMBBI);
+  case LoongArch::PseudoCmpXchg128:
+  case LoongArch::PseudoCmpXchg128Acquire:
+    return expandAtomicCmpXchg128(MBB, MBBI, NextMBBI);
   case LoongArch::PseudoMaskedCmpXchg32:
     return expandAtomicCmpXchg(MBB, MBBI, true, 32, NextMBBI);
   case LoongArch::PseudoMaskedAtomicLoadMax32:
@@ -602,6 +608,107 @@ bool LoongArchExpandAtomicPseudo::expandAtomicCmpXchg(
   return true;
 }
 
+bool LoongArchExpandAtomicPseudo::expandAtomicCmpXchg128(
+    MachineBasicBlock &MBB, MachineBasicBlock::iterator MBBI,
+    MachineBasicBlock::iterator &NextMBBI) {
+  MachineInstr &MI = *MBBI;
+  DebugLoc DL = MI.getDebugLoc();
+  MachineFunction *MF = MBB.getParent();
+  auto LoopHeadMBB = MF->CreateMachineBasicBlock(MBB.getBasicBlock());
+  auto LoopTailMBB = MF->CreateMachineBasicBlock(MBB.getBasicBlock());
+  auto TailMBB = MF->CreateMachineBasicBlock(MBB.getBasicBlock());
+  auto DoneMBB = MF->CreateMachineBasicBlock(MBB.getBasicBlock());
+
+  // Insert new MBBs
+  MF->insert(++MBB.getIterator(), LoopHeadMBB);
+  MF->insert(++LoopHeadMBB->getIterator(), LoopTailMBB);
+  MF->insert(++LoopTailMBB->getIterator(), TailMBB);
+  MF->insert(++TailMBB->getIterator(), DoneMBB);
+
+  // Set up successors and transfer remaining instructions to DoneMBB.
+  LoopHeadMBB->addSuccessor(LoopTailMBB);
+  LoopHeadMBB->addSuccessor(TailMBB);
+  LoopTailMBB->addSuccessor(DoneMBB);
+  LoopTailMBB->addSuccessor(LoopHeadMBB);
+  TailMBB->addSuccessor(DoneMBB);
+  DoneMBB->splice(DoneMBB->end(), &MBB, MI, MBB.end());
+  DoneMBB->transferSuccessors(&MBB);
+  MBB.addSuccessor(LoopHeadMBB);
+
+  Register DestLoReg = MI.getOperand(0).getReg();
+  Register DestHiReg = MI.getOperand(1).getReg();
+  Register ScratchReg = MI.getOperand(2).getReg();
+  Register AddrReg = MI.getOperand(3).getReg();
+  Register CmpValLoReg = MI.getOperand(4).getReg();
+  Register CmpValHiReg = MI.getOperand(5).getReg();
+  Register NewValLoReg = MI.getOperand(6).getReg();
+  Register NewValHiReg = MI.getOperand(7).getReg();
+
+    // .loophead:
+    //   ll.d res_lo, (addr)
+    //   ld.d res_hi, (addr), 8
+    //   bne dest_lo, cmpval_lo, tail
+    //   bne dest_hi, cmpval_hi, tail
+  BuildMI(LoopHeadMBB, DL, TII->get(LoongArch::LL_D), DestLoReg)
+      .addReg(AddrReg)
+      .addImm(0);
+  BuildMI(LoopHeadMBB, DL, TII->get(LoongArch::LD_D), DestHiReg)
+      .addReg(AddrReg)
+      .addImm(8);
+  BuildMI(LoopHeadMBB, DL, TII->get(LoongArch::BNE))
+      .addReg(DestLoReg)
+      .addReg(CmpValLoReg)
+      .addMBB(TailMBB);
+  BuildMI(...
[truncated]

tangaac · 2024-11-19T09:42:13Z

ir

define void @cmpxchg_i128_acquire_monotonic(ptr %ptr, i128 %cmp, i128 %val) nounwind {
  %res = cmpxchg ptr %ptr, i128 %cmp, i128 %val acquire monotonic
  ret void
}

asm

cmpxchg_i128_acquire_monotonic:
# %bb.0:
.LBB1_1: # =>This Inner Loop Header: Depth=1
  ll.d $a5, $a0, 0
  dbar 20
  ld.d $a6, $a0, 8
  bne $a5, $a1, .LBB1_3
  bne $a6, $a2, .LBB1_3
# %bb.2: # in Loop: Header=BB1_1 Depth=1
  move $a7, $a3
  sc.q $a7, $a4, $a0
  beqz $a7, .LBB1_1
  b .LBB1_4
.LBB1_3:
  dbar 20
.LBB1_4:
  ret

github-actions · 2024-11-19T09:42:36Z

✅ With the latest revision this PR passed the C/C++ code formatter.

heiher

In addition, I propose adding test cases for atomic ops based on the native cmpxchg128. Due to the lack of the ll.q instruction, even atomic128_read needs to be implemented using cmpxchg128 (ll.d + ld.d + sc.q). Alternatively, a vector 128-bit load can be used if lsx is available.

llvm/test/CodeGen/LoongArch/ir-instruction/atomic-cmpxchg-128.ll

SixWeining · 2024-12-02T03:30:36Z

llvm/test/CodeGen/LoongArch/ir-instruction/atomic-cmpxchg-128.ll

Add --verify-machineinstrs option as the code calls lots of BuildMI(...).

llvm/lib/Target/LoongArch/LoongArchExpandAtomicPseudoInsts.cpp

heiher

LGTM. Thanks.

SixWeining · 2025-01-23T03:03:36Z

llvm/lib/TargetParser/Host.cpp

@@ -2135,12 +2135,12 @@ const StringMap<bool> sys::getHostCPUFeatures() {
  Features["div32"] = cpucfg2 & (1U << 26);   // CPUCFG.2.DIV32
  Features["lam-bh"] = cpucfg2 & (1U << 27);  // CPUCFG.2.LAM_BH
  Features["lamcas"] = cpucfg2 & (1U << 28);  // CPUCFG.2.LAMCAS
+  Features["scq"] = cpucfg2 & (1U << 30);     // CPUCFG.2.SCQLL


SCQLL: Did you mean SCQ ?

llvmbot added clang Clang issues not falling into any other category clang:driver 'clang' and 'clang++' user-facing binaries. Not 'clang-cl' clang:frontend Language frontend issues, e.g. anything involving "Sema" backend:loongarch labels Nov 19, 2024

tangaac force-pushed the feature/scq branch from 1b5ca04 to 3181959 Compare November 19, 2024 09:44

tangaac force-pushed the feature/scq branch 2 times, most recently from 7eb12cd to b8d1a19 Compare November 28, 2024 03:25

SixWeining requested a review from heiher November 28, 2024 07:30

heiher reviewed Nov 28, 2024

View reviewed changes

llvm/test/CodeGen/LoongArch/ir-instruction/atomic-cmpxchg-128.ll Show resolved Hide resolved

llvm/test/CodeGen/LoongArch/ir-instruction/atomic-cmpxchg-128.ll Show resolved Hide resolved

SixWeining reviewed Jan 22, 2025

View reviewed changes

tangaac force-pushed the feature/scq branch from 884776b to 1cf7df8 Compare January 22, 2025 06:06

heiher reviewed Jan 22, 2025

View reviewed changes

llvm/lib/Target/LoongArch/LoongArchExpandAtomicPseudoInsts.cpp Show resolved Hide resolved

tangaac added 5 commits January 22, 2025 18:16

[LoongArch] Support sc.q instruction for 128bit cmpxchg operation

ad154c5

scq work with ld-seq-sa

e28396f

small change

09519f9

add --verify-machineinstrs option for cmpxchg-128 test

18d051b

fix comment for missing dbar acquire

d07502b

tangaac force-pushed the feature/scq branch from f52cc79 to d07502b Compare January 22, 2025 10:20

format code

d1c2506

heiher approved these changes Jan 23, 2025

View reviewed changes

SixWeining reviewed Jan 23, 2025

View reviewed changes

fix a comment

2fa73ab

SixWeining approved these changes Jan 23, 2025

View reviewed changes

SixWeining merged commit 19834b4 into llvm:main Jan 23, 2025
6 of 7 checks passed

tangaac deleted the feature/scq branch February 8, 2025 01:24

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[LoongArch] Support sc.q instruction for 128bit cmpxchg operation #116771

[LoongArch] Support sc.q instruction for 128bit cmpxchg operation #116771

Uh oh!

tangaac commented Nov 19, 2024

Uh oh!

llvmbot commented Nov 19, 2024 •

edited

Loading

Uh oh!

tangaac commented Nov 19, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Nov 19, 2024 •

edited

Loading

Uh oh!

heiher left a comment

Uh oh!

Uh oh!

Uh oh!

SixWeining Dec 2, 2024

Uh oh!

Uh oh!

heiher left a comment

Uh oh!

SixWeining Jan 23, 2025

Uh oh!

Uh oh!

Uh oh!

[LoongArch] Support sc.q instruction for 128bit cmpxchg operation #116771

[LoongArch] Support sc.q instruction for 128bit cmpxchg operation #116771

Uh oh!

Conversation

tangaac commented Nov 19, 2024

Uh oh!

llvmbot commented Nov 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tangaac commented Nov 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Nov 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

heiher left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

SixWeining Dec 2, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

heiher left a comment

Choose a reason for hiding this comment

Uh oh!

SixWeining Jan 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

llvmbot commented Nov 19, 2024 •

edited

Loading

tangaac commented Nov 19, 2024 •

edited

Loading

github-actions bot commented Nov 19, 2024 •

edited

Loading