Skip to content

Commit e628a14

Browse files
delnerdelner
authored
Merge pull request #1305 from DataDog/ci/integration_suite
Add integration test suite
2 parents 8398dd7 + b934675 commit e628a14

File tree

112 files changed

+2957
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

112 files changed

+2957
-0
lines changed

‎.circleci/config.yml

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,34 @@ orbs:
162162
key: bundle-{{ .Environment.CIRCLE_CACHE_VERSION }}-<<parameters.ruby_version>>-{{ checksum "lib/ddtrace/version.rb" }}-{{ .Branch }}-{{ checksum "Gemfile" }}-{{ checksum "Appraisals" }}-{{ checksum "ddtrace.gemspec" }}-{{ checksum ".circleci/bundle_checksum" }}'
163163
paths:
164164
- /usr/local/bundle
165+
build_and_test_integration:
166+
executor: docker-integration
167+
parameters:
168+
ruby_version:
169+
description: Ruby version
170+
type: string
171+
integration_apps:
172+
description: Integration apps
173+
type: string
174+
steps:
175+
- checkout
176+
- setup_remote_docker:
177+
docker_layer_caching: true
178+
- run:
179+
name: Build integration base images
180+
command: ./integration/script/build-images -v <<parameters.ruby_version>>
181+
- run:
182+
name: Build integration app images
183+
command: |
184+
for i in <<parameters.integration_apps>>; do
185+
./integration/apps/$i/script/build-images -v <<parameters.ruby_version>>
186+
done
187+
- run:
188+
name: Test integration apps
189+
command: |
190+
for i in <<parameters.integration_apps>>; do
191+
./integration/apps/$i/script/ci -v <<parameters.ruby_version>>
192+
done
165193
test:
166194
<<: *test_job_default
167195
docker:
@@ -300,6 +328,9 @@ orbs:
300328
name: Wait for container on port <<parameters.port>>
301329
command: dockerize -wait 'tcp://localhost:<<parameters.port>>' -timeout '<<parameters.timeout>>'
302330
executors:
331+
docker-integration:
332+
docker:
333+
- image: circleci/buildpack-deps:stretch
303334

304335
jobs:
305336
"deploy release":
@@ -439,6 +470,53 @@ workflows:
439470
filters:
440471
branches:
441472
only: /bump_to_version_.*/
473+
# Integration
474+
- orb/build_and_test_integration:
475+
name: build_and_test_integration-2.0
476+
integration_apps: 'rack'
477+
ruby_version: '2.0'
478+
<<: *filters_all_branches_and_tags
479+
- orb/build_and_test_integration:
480+
name: build_and_test_integration-2.1
481+
integration_apps: 'rack'
482+
ruby_version: '2.1'
483+
<<: *filters_all_branches_and_tags
484+
- orb/build_and_test_integration:
485+
name: build_and_test_integration-2.2
486+
integration_apps: 'rack'
487+
ruby_version: '2.2'
488+
<<: *filters_all_branches_and_tags
489+
- orb/build_and_test_integration:
490+
name: build_and_test_integration-2.3
491+
integration_apps: 'rack rails-five'
492+
ruby_version: '2.3'
493+
<<: *filters_all_branches_and_tags
494+
- orb/build_and_test_integration:
495+
name: build_and_test_integration-2.4
496+
integration_apps: 'rack rails-five'
497+
ruby_version: '2.4'
498+
<<: *filters_all_branches_and_tags
499+
- orb/build_and_test_integration:
500+
name: build_and_test_integration-2.5
501+
integration_apps: 'rack rails-five'
502+
ruby_version: '2.5'
503+
<<: *filters_all_branches_and_tags
504+
- orb/build_and_test_integration:
505+
name: build_and_test_integration-2.6
506+
integration_apps: 'rack rails-five'
507+
ruby_version: '2.6'
508+
<<: *filters_all_branches_and_tags
509+
- orb/build_and_test_integration:
510+
name: build_and_test_integration-2.7
511+
integration_apps: 'rack rails-five'
512+
ruby_version: '2.7'
513+
<<: *filters_all_branches_and_tags
514+
# TODO: Get apps working with Ruby 3.0
515+
# - orb/build_and_test_integration:
516+
# name: build_and_test_integration-3.0
517+
# integration_apps: 'rack rails-five'
518+
# ruby_version: '3.0'
519+
# <<: *filters_all_branches_and_tags
442520
# MRI
443521
- orb/build:
444522
<<: *config-2_0

‎integration/README.md

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
# Datadog APM Ruby integration test suite
2+
3+
Integration tests for `ddtrace` that use a variety of real applications.
4+
5+
## Quickstart
6+
7+
1. Build Docker base images:
8+
9+
```sh
10+
#!/bin/bash
11+
./script/build-images
12+
```
13+
14+
2. Choose an application and follow instructions (in corresponding `README.md`.)
15+
16+
## Demo applications
17+
18+
Ruby demo applications are configured with Datadog APM, which can be used to generate sample traces/profiles. These are used to drive tests in the integration suite.
19+
20+
### Applications
21+
22+
See `README.md` in each directory for more information:
23+
24+
- `apps/rack`: Rack application
25+
- `apps/rails-five`: Rails 5 application
26+
27+
### Base images
28+
29+
The `images/` folders hosts some images for Ruby applications.
30+
31+
- `datadog/dd-apm-demo:wrk` / `images/wrk/Dockerfile`: `wrk` load testing application (for generating load)
32+
- `datadog/dd-apm-demo:agent` / `images/agent/Dockerfile`: Datadog agent (with default configuration)
33+
- `datadog/dd-apm-demo:rb-<RUBY_VERSION>` / `images/<RUBY_VERSION>/Dockerfile`: MRI Ruby & `Datadog::DemoEnv` (where `<RUBY_VERSION>` is minor version e.g. `2.7`)
34+
35+
Ruby base images include `Datadog::DemoEnv` and other helpers.
36+
37+
### Debugging
38+
39+
#### Profiling memory
40+
41+
Create a memory heap dump via:
42+
43+
```sh
44+
# Profile for 5 minutes, dump heap to /data/app/ruby-heap.dump
45+
# Where PID = process ID
46+
bundle exec rbtrace -p PID -e 'Thread.new{GC.start; require "objspace"; ObjectSpace.trace_object_allocations_start; sleep(300); io=File.open("/data/app/ruby-heap.dump", "w"); ObjectSpace.dump_all(output: io); io.close}'
47+
```
48+
49+
Then analyze it using `analyzer.rb` (built into the Ruby base images) with:
50+
51+
```sh
52+
# Group objects by generation
53+
ruby /vendor/dd-demo/datadog/analyzer.rb /data/app/ruby-heap.dump
54+
55+
# List objects in GEN_NUM, group by source location
56+
ruby /vendor/dd-demo/datadog/analyzer.rb /data/app/ruby-heap.dump GEN_NUM
57+
58+
# List objects in all generations, group by source location, descending.
59+
ruby /vendor/dd-demo/datadog/analyzer.rb /data/app/ruby-heap.dump objects
60+
61+
# List objects in all generations, group by source location, descending.
62+
# Up to generation END_GEN.
63+
ruby /vendor/dd-demo/datadog/analyzer.rb /data/app/ruby-heap.dump objects END_GEN
64+
65+
# List objects in all generations, group by source location, descending.
66+
# Between generations START_GEN to END_GEN inclusive.
67+
ruby /vendor/dd-demo/datadog/analyzer.rb /data/app/ruby-heap.dump objects END_GEN START_GEN
68+
```

‎integration/apps/rack/.dockerignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Gemfile.lock

‎integration/apps/rack/.envrc.sample

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
export DD_API_KEY=<Your Datadog API key here>

‎integration/apps/rack/.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# Ignore local variables
2+
.envrc
3+
.byebug_history
4+
data

‎integration/apps/rack/.rspec

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
--require spec_helper

‎integration/apps/rack/Dockerfile

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
# Select base image
2+
ARG BASE_IMAGE
3+
FROM ${BASE_IMAGE}
4+
5+
# Setup directory
6+
RUN mkdir /app
7+
WORKDIR /app
8+
9+
# Setup specific version of ddtrace, if specified.
10+
ARG ddtrace_git
11+
ENV DD_DEMO_ENV_GEM_GIT_DDTRACE ${ddtrace_git}
12+
13+
ARG ddtrace_ref
14+
ENV DD_DEMO_ENV_GEM_REF_DDTRACE ${ddtrace_ref}
15+
16+
# Install dependencies
17+
COPY Gemfile /app/Gemfile
18+
RUN bundle install
19+
20+
# Add files
21+
COPY . /app
22+
23+
# Set entrypoint
24+
ENTRYPOINT ["/bin/bash", "-c"]
25+
CMD ["bin/setup && bin/run"]

‎integration/apps/rack/Dockerfile-ci

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Select base image
2+
ARG BASE_IMAGE
3+
FROM ${BASE_IMAGE}
4+
5+
# Add gem
6+
COPY . /vendor/dd-trace-rb
7+
8+
# Install dependencies
9+
# Setup specific version of ddtrace, if specified.
10+
ENV DD_DEMO_ENV_GEM_LOCAL_DDTRACE /vendor/dd-trace-rb
11+
RUN bundle install

‎integration/apps/rack/Gemfile

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
require 'datadog/demo_env'
2+
3+
source 'https://rubygems.org' do
4+
gem 'puma'
5+
gem 'unicorn'
6+
7+
gem 'rack'
8+
gem 'dogstatsd-ruby'
9+
10+
gem 'ffi'
11+
gem 'google-protobuf'
12+
13+
# Choose correct specs for 'ddtrace' demo environment
14+
gem 'ddtrace', *Datadog::DemoEnv.gem_spec('ddtrace')
15+
16+
# Development
17+
gem 'byebug'
18+
# gem 'pry-stack_explorer', platform: :ruby
19+
# gem 'rbtrace'
20+
# gem 'ruby-prof'
21+
22+
# Testing/CI
23+
gem 'rspec'
24+
end

‎integration/apps/rack/README.md

Lines changed: 94 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,94 @@
1+
# Rack: Demo application for Datadog APM
2+
3+
A generic Rack web application with some common use scenarios.
4+
5+
For generating Datadog APM traces and profiles.
6+
7+
## Installation
8+
9+
Install [direnv](https://github.com/direnv/direnv) for applying local settings.
10+
11+
1. `cp .envrc.sample .envrc` and add your Datadog API key.
12+
2. `direnv allow` to load the env var.
13+
4. `docker-compose run --rm app bin/setup`
14+
15+
## Running the application
16+
17+
### To monitor performance of Docker containers with Datadog
18+
19+
```sh
20+
docker run --rm --name dd-agent -v /var/run/docker.sock:/var/run/docker.sock:ro -v /proc/:/host/proc/:ro -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro -e API_KEY=$DD_API_KEY datadog/docker-dd-agent:latest
21+
```
22+
23+
### Starting the web server
24+
25+
Run `docker-compose up` to auto-start the webserver. It should bind to `localhost:80`.
26+
27+
Alternatively, you can run it manually with:
28+
29+
```sh
30+
docker-compose run --rm -p 80:80 app bin/run <process>
31+
```
32+
33+
The `<process>` argument is optional, and will default to `DD_DEMO_ENV_PROCESS` if not provided. See [Processes](#processes) for more details.
34+
35+
##### Processes
36+
37+
Within the container, run `bin/run <process>` where `<process>` is one of the following values:
38+
39+
- `puma`: Puma web server
40+
- `unicorn`: Unicorn web server
41+
- `console`: Rails console
42+
- `irb`: IRB session
43+
44+
Alternatively, set `DD_DEMO_ENV_PROCESS` to run a particular process by default when `bin/run` is run.
45+
46+
##### Features
47+
48+
Set `DD_DEMO_ENV_PROCESS` to a comma-delimited list of any of the following values to activate the feature:
49+
50+
- `tracing`: Tracing instrumentation
51+
- `profiling`: Profiling (NOTE: Must also set `DD_PROFILING_ENABLED` to match.)
52+
- `debug`: Enable diagnostic debug mode
53+
- `analytics`: Enable trace analytics
54+
- `runtime_metrics`: Enable runtime metrics
55+
- `pprof_to_file`: Dump profiling pprof to file instead of agent.
56+
57+
e.g. `DD_DEMO_ENV_PROCESS=tracing,profiling`
58+
59+
##### Routes
60+
61+
```sh
62+
# Health check
63+
curl -v localhost/health
64+
65+
# Basic demo scenarios
66+
curl -v localhost/basic/fibonacci
67+
curl -v -XPOST localhost/basic/default
68+
69+
# Job demo scenarios
70+
curl -v -XPOST localhost/jobs
71+
```
72+
73+
### Load tester
74+
75+
Docker configuration automatically creates and runs [Wrk](https://github.com/wg/wrk) load testing containers. By default it runs the `basic/default` scenario described in the `wrk` image to give a baseload.
76+
77+
You can modify the `load-tester` container in `docker-compose.yml` to change the load type or scenario run. Set the container's `command` to any set of arguments `wrk` accepts.
78+
79+
You can also define your own custom scenario by creating a LUA file, mounting it into the container, and passing it as an argument via `command`.
80+
81+
### Running integration tests
82+
83+
You can run integration tests using the following and substituting for the Ruby major and minor version (e.g. `2.7`)
84+
85+
```sh
86+
./script/build-images -v <RUBY_VERSION>
87+
./script/ci -v <RUBY_VERSION>
88+
```
89+
90+
Or inside a running container:
91+
92+
```sh
93+
./bin/test
94+
```

0 commit comments

Comments
 (0)