Implement Manifest V3 objects and deployment

.gitignore

@@ -1,9 +1,11 @@
 .idea/libraries
+.idea/jarRepositories.xml
 .idea/tasks.xml
 .idea/workspace.xml
 target/
 bin/
 .project
 .classpath
 .settings/
-.factorypath
+.factorypath
+.gradle

README.md

@@ -8,24 +8,14 @@
 | `cloudfoundry-operations`     | [![javadoc](https://javadoc.io/badge2/org.cloudfoundry/cloudfoundry-operations/javadoc.svg)](https://javadoc.io/doc/org.cloudfoundry/cloudfoundry-operations)
 | `cloudfoundry-util`           | [![javadoc](https://javadoc.io/badge2/org.cloudfoundry/cloudfoundry-util/javadoc.svg)](https://javadoc.io/doc/org.cloudfoundry/cloudfoundry-util)
 
-| Job | Status
-| --- | ------
-| `unit-test`             | [![unit-test-master](https://java-experience.ci.springapps.io/api/v1/teams/java-experience/pipelines/java-client/jobs/unit-test-master/badge)](https://java-experience.ci.springapps.io/teams/java-experience/pipelines/java-client/jobs/unit-test-master)
-| `integration-test-2.5`  | [![integration-test-2.5-master](https://java-experience.ci.springapps.io/api/v1/teams/java-experience/pipelines/java-client/jobs/integration-test-2.5-master/badge)](https://java-experience.ci.springapps.io/teams/java-experience/pipelines/java-client/jobs/integration-test-2.5-master)
-| `integration-test-2.6`  | [![integration-test-2.6-master](https://java-experience.ci.springapps.io/api/v1/teams/java-experience/pipelines/java-client/jobs/integration-test-2.6-master/badge)](https://java-experience.ci.springapps.io/teams/java-experience/pipelines/java-client/jobs/integration-test-2.6-master)
-| `integration-test-2.7`  | [![integration-test-2.7-master](https://java-experience.ci.springapps.io/api/v1/teams/java-experience/pipelines/java-client/jobs/integration-test-2.7-master/badge)](https://java-experience.ci.springapps.io/teams/java-experience/pipelines/java-client/jobs/integration-test-2.7-master)
-| `integration-test-2.8`  | [![integration-test-2.8-master](https://java-experience.ci.springapps.io/api/v1/teams/java-experience/pipelines/java-client/jobs/integration-test-2.8-master/badge)](https://java-experience.ci.springapps.io/teams/java-experience/pipelines/java-client/jobs/integration-test-2.8-master)
-| `deploy`                | [![deploy-master](https://java-experience.ci.springapps.io/api/v1/teams/java-experience/pipelines/java-client/jobs/deploy-master/badge)](https://java-experience.ci.springapps.io/teams/java-experience/pipelines/java-client/jobs/deploy-master)
-
-
 The `cf-java-client` project is a Java language binding for interacting with a Cloud Foundry instance.  The project is broken up into a number of components that expose different levels of abstraction depending on need.
 
 * `cloudfoundry-client` – Interfaces, request, and response objects mapping to the [Cloud Foundry REST APIs][a].  This project has no implementation and therefore cannot connect to a Cloud Foundry instance on its own.
-* `cloudfoundry-client-reactor` – The default implementation of the `cloudfoundry-client` project.  This implementation is based on the Reactor Netty [`HttpClient`][h].
+* `cloudfoundry-client-reactor` – The default implementation of the `cloudfoundry-client` project.  This implementation is based on Reactor Netty [`HttpClient`][h].
 * `cloudfoundry-operations` – An API and implementation that corresponds to the [Cloud Foundry CLI][c] operations.  This project builds on the `cloudfoundry-client` and therefore has a single implementation.
 
 ## Versions
-The Cloud Foundry Java Client has two active versions. The `4.x` line uses Spring Boot `2.2.x` just to manage its dependencies, while the `3.x` line uses Spring Boot `2.1.x`. Unless you have a specific dependency-related reason for using the older version we recommend you adopt the `4.x` line.
+The Cloud Foundry Java Client has two active versions. The `5.x` line is compatible with Spring Boot `2.4.x - 2.6.x` just to manage its dependencies, while the `4.x` line uses Spring Boot `2.3.x`.
 
 ## Dependencies
 Most projects will need two dependencies; the Operations API and an implementation of the Client API.  For Maven, the dependencies would be defined like this:
@@ -35,13 +25,12 @@ Most projects will need two dependencies; the Operations API and an implementati
     <dependency>
         <groupId>org.cloudfoundry</groupId>
         <artifactId>cloudfoundry-client-reactor</artifactId>
-        <version>4.3.0.RELEASE</version>
+        <version>latest.RELEASE</version>
     </dependency>
     <dependency>
         <groupId>org.cloudfoundry</groupId>
         <artifactId>cloudfoundry-operations</artifactId>
-        <version>4.3.0.RELEASE</version>
-    </dependency>
+        <version>latest.RELEASE</version>
     </dependency>
     ...
 </dependencies>
@@ -67,8 +56,8 @@ For Gradle, the dependencies would be defined like this:
 
 ```groovy
 dependencies {
-    compile 'org.cloudfoundry:cloudfoundry-client-reactor:4.3.0.RELEASE'
-    compile 'org.cloudfoundry:cloudfoundry-operations:4.3.0.RELEASE'
+    compile 'org.cloudfoundry:cloudfoundry-client-reactor:<latest>.RELEASE'
+    compile 'org.cloudfoundry:cloudfoundry-operations:<latest>.RELEASE'
     ...
 }
 ```
@@ -208,7 +197,7 @@ Once you've got a reference to the `CloudFoundryOperations`, it's time to start
 
 1. Requests a list of all organizations
 1. Extracts the name of each organization
-1. Prints the name of the each organization to `System.out`
+1. Prints the name of each organization to `System.out`
 
 ```java
 cloudFoundryOperations.organizations()
@@ -245,6 +234,24 @@ The above example is more complicated:
 1. `.flatMapIterable(...)` – Substitutes the original `Mono` with a `Flux` of the `Resource`s returned by the requested page.
 1. `.map(...)` – Maps the `Resource` to an `OrganizationSummary` type.
 
+## Troubleshooting
+
+If you are having issues with the cf-java-client in your applications...
+
+First, [read this article on debugging reactive applications](https://spring.io/blog/2019/03/28/reactor-debugging-experience) and watch this [Spring Tips Video](https://spring.io/blog/2019/05/29/spring-tips-debugging-reactor-applications) also on debugging reactive apps.
+
+Beyond that, it is helpful to capture the following information:
+
+1. The version of cf-java-client you are using
+2. If you are using Spring Boot & if so, the version
+3. A description of the problem behavior.
+4. A list of things, if any, you have recently changed in your project (even if seemingly unrelated)
+5. If you are getting failures with an operation or client request, capture the request and response information.
+    - You may capture basic request and response information by setting the log level for `cloudfoundry-client` to `DEBUG`. For example with Spring Boot, set `logging.level.cloudfoundry-client=debug`.
+    - You may perform a wire capture by setting the log level for `cloudfoundry-client.wire` to `TRACE`. For example with Spring Boot, set `logging.level.cloudfoundry-client.wire=trace`.
+
+If you open a Github issue with a request for help, please include as much of the information above as possible and do not forget to sanitize any request/response data posted.
+
 ## Development
 The project depends on Java 8. To build from source and install to your local Maven cache, run the following:
 
@@ -261,8 +268,20 @@ To run the integration tests, run the following:
 $ ./mvnw -Pintegration-test clean test
 ```
 
+To run a specific integration test, run the following replacing with `-Dtest=org.cloudfoundry.TestClass#test-method` (`#test-method` is optional):
+
+```shell
+./mvnw -Pintegration-test clean test -Dtest=org.cloudfoundry.client.v3.ServiceBrokersTest#update
+```
+
+To run tests & enable HTTP trace output, execute:
+
+```shell
+CLIENT_LOGGING_LEVEL=trace ./mvnw -Pintegration-test clean test
+```
+
 **IMPORTANT**
-Integration tests should be run against an empty Cloud Foundry instance. The integration tests are destructive, affecting nearly everything on an instance given the chance.
+Integration tests require admin access and should be run against an empty Cloud Foundry instance. The integration tests are destructive, affecting nearly everything on an instance given the chance.
 
 The integration tests require a running instance of Cloud Foundry to test against. To configure the integration tests with the appropriate connection information use the following environment variables:
 
@@ -279,6 +298,14 @@ Name | Description
 `TEST_PROXY_USERNAME` | _(Optional)_ The username for a proxy to route all requests through
 `TEST_SKIPSSLVALIDATION` | _(Optional)_ Whether to skip SSL validation when connecting to the Cloud Foundry instance.  Defaults to `false`.
 
+If you do not have access to a CloudFoundry instance with admin access, you can run one locally using [bosh-deployment](https://github.com/cloudfoundry/bosh-deployment) & [cf-deployment](https://github.com/cloudfoundry/cf-deployment/) and Virtualbox.
+
+For instructions installing Bosh in VirtualBox using `bosh-deployment`, see the [Install Section to install Bosh](https://bosh.io/docs/bosh-lite/).
+
+With Bosh installed, follow the [deployment guide to get CF installed](https://github.com/cloudfoundry/cf-deployment/blob/main/texts/deployment-guide.md). You can skip to step 4, since you're installing into VirtualBox. Be sure to read the entire document before you begin, however pay specific attention to [this section which has specific instructions for running locally](https://github.com/cloudfoundry/cf-deployment/blob/main/texts/deployment-guide.md#for-operators-deploying-cf-to-local-bosh-lite).
+
+Lastly before running the tests, it is strongly recommended that you take a snapshot of the VMs in the environment. This allows for the quick rollback of the environment should the tests break something (they don't generally, integration tests should clean up after themselves).
+
 ## Contributing
 [Pull requests][u] and [Issues][e] are welcome.
 

ci/Dockerfile

@@ -1,7 +1,14 @@
-FROM ubuntu:bionic
+ARG base_image=openjdk:8-jre
+FROM ${base_image}
+
+ADD https://repo.spring.io/libs-release/io/spring/concourse/releasescripts/concourse-release-scripts/0.3.2/concourse-release-scripts-0.3.2.jar /opt/concourse-release-scripts.jar
 
 RUN apt-get update && apt-get install --no-install-recommends -y \
     ca-certificates \
     curl \
     jq \
- && rm -rf /var/lib/apt/lists/*
+    gnupg \
+    && rm -rf /var/lib/apt/lists/*
+
+RUN curl -fL https://getcli.jfrog.io | sh && \
+    mv jfrog /usr/local/bin/

ci/claim-environment.sh

@@ -0,0 +1,86 @@
+#!/usr/bin/env bash
+
+set -euo pipefail
+
+ROOT=$(realpath "$(dirname "${BASH_SOURCE[0]}")"/../..)
+
+if [[ -d "${ROOT}"/om ]]; then
+  printf "➜ Expanding om\n"
+  tar xzf "${ROOT}"/om/om-linux-*.tar.gz -C "${ROOT}"/om
+  export PATH="${ROOT}"/om:${PATH}
+fi
+
+printf "Claiming environment from %s\n" "${POOL}"
+
+CLAIM=$(curl \
+  --fail \
+  --location \
+  --show-error \
+  --silent \
+  --header 'Accept: application/json' \
+  --request "POST" \
+  "https://environments.toolsmiths.cf-app.com/pooled_gcp_engineering_environments/claim?api_token=${API_TOKEN}&pool_name=${POOL}&notes=Claimed%20by%20Java%20Buildpack%20CI")
+
+printf "Claimed %s\n" "$(jq -n -r --argjson claim "${CLAIM}" '$claim.name')"
+
+CREDENTIALS=$(om \
+  --target "$(jq -n -r --argjson claim "${CLAIM}" '$claim.ops_manager.url')" \
+  --username "$(jq -n -r --argjson claim "${CLAIM}" '$claim.ops_manager.username')" \
+  --password "$(jq -n -r --argjson claim "${CLAIM}" '$claim.ops_manager.password')" \
+  credentials \
+  --product-name cf \
+  --credential-reference .uaa.admin_credentials \
+  --format json)
+
+  jq \
+    -n -r \
+    --argjson claim "${CLAIM}" \
+    --argjson credentials "${CREDENTIALS}" \
+    '{ name: $claim.name, username: $credentials.identity, password: $credentials.password }' \
+    > "${ROOT}"/environment/cf-creds.json
+
+
+CREDENTIALS=$(om \
+  --target "$(jq -n -r --argjson claim "${CLAIM}" '$claim.ops_manager.url')" \
+  --username "$(jq -n -r --argjson claim "${CLAIM}" '$claim.ops_manager.username')" \
+  --password "$(jq -n -r --argjson claim "${CLAIM}" '$claim.ops_manager.password')" \
+  credentials \
+  --product-name cf \
+  --credential-reference .uaa.admin_client_credentials \
+  --format json)
+
+  jq \
+    -n -r \
+    --argjson claim "${CLAIM}" \
+    --argjson credentials "${CREDENTIALS}" \
+    '{ client: $credentials.identity, secret: $credentials.password }' \
+    > "${ROOT}"/environment/uaa-creds.json
+
+printf "Patching for TCP Routing Support\n"
+
+om \
+  --target "$(jq -n -r --argjson claim "${CLAIM}" '$claim.ops_manager.url')" \
+  --username "$(jq -n -r --argjson claim "${CLAIM}" '$claim.ops_manager.username')" \
+  --password "$(jq -n -r --argjson claim "${CLAIM}" '$claim.ops_manager.password')" \
+  staged-config \
+  --product-name cf > /tmp/cf.yml
+
+TCP_ROUTES_LB="$(jq -n -r --argjson claim "${CLAIM}" '$claim.tcp_router_pool')"
+
+om \
+  --target "$(jq -n -r --argjson claim "${CLAIM}" '$claim.ops_manager.url')" \
+  --username "$(jq -n -r --argjson claim "${CLAIM}" '$claim.ops_manager.username')" \
+  --password "$(jq -n -r --argjson claim "${CLAIM}" '$claim.ops_manager.password')" \
+  configure-product \
+  --config /tmp/cf.yml \
+  --ops-file cf-java-client/ci/tcp-routes.yml \
+  --var TCP_ROUTES_LB="tcp:${TCP_ROUTES_LB}"
+
+om \
+  --target "$(jq -n -r --argjson claim "${CLAIM}" '$claim.ops_manager.url')" \
+  --username "$(jq -n -r --argjson claim "${CLAIM}" '$claim.ops_manager.username')" \
+  --password "$(jq -n -r --argjson claim "${CLAIM}" '$claim.ops_manager.password')" \
+  apply-changes \
+  -n cf
+
+printf "Environment updated with TCP Routes\n"

ci/deploy.sh

@@ -4,6 +4,28 @@ set -euo pipefail
 
 [[ -d $PWD/maven && ! -d $HOME/.m2 ]] && ln -s $PWD/maven $HOME/.m2
 
+function clean_gpg {
+    FINGERPRINT=$(gpg --list-keys | head -4 | tail -1 | tr -d ' ')
+    gpg --batch --yes --delete-secret-keys "$FINGERPRINT"
+    gpg --batch --yes --delete-keys "$FINGERPRINT"
+}
+
+trap clean_gpg EXIT
+gpg --batch --import-options import-show --import <(echo "$MAVEN_GPG_PRIVATE_KEY")
+
+mkdir -p ~/.m2
+cat <<EOF > ~/.m2/settings.xml
+<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
+  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
+  xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 https://maven.apache.org/xsd/settings-1.0.0.xsd">
+  <servers>
+    <server>
+      <id>gpg.passphrase</id>
+      <passphrase>${MAVEN_GPG_PASSPHRASE}</passphrase>
+    </server>
+  </servers>
+</settings>
+EOF
+
 cd cf-java-client
 ./mvnw -q -Dmaven.test.skip=true deploy
-cp target/build-info.json ../build-info/build-info.json