@@ -251,6 +251,36 @@ These results can also be placed into a PostgreSQL database by setting the
`RSPEC_PROFILING_POSTGRES_URL` variable. This is used to profile the test suite
when running in the CI environment.
## Memory profiling
One of the reasons of the increased memory footprint could be Ruby memory fragmentation.
To diagnose it, you can visualize Ruby heap as described in [this post by Aaron Patterson](https://tenderlovemaking.com/2017/09/27/visualizing-your-ruby-heap.html).
To start, you want to dump the heap of the process you are investigating to a JSON file.
You need to run the command inside the process you are exploring, you may do that with `rbtrace`.
`rbtrace` is already present in GitLab `Gemfile`, you just need to require it.
It could be achieved running webserver or Sidekiq with the environment variable set to `ENABLE_RBTRACE=1`.
Having the JSON, you finally could render a picture using the script [provided by Aaron](https://gist.github.com/tenderlove/f28373d56fdd03d8b514af7191611b88) or similar:
```sh
ruby heapviz.rb heap.json
```
Fragmented Ruby heap snapshot could look like this:
Memory fragmentation could be reduced by tuning GC parameters as described in [this post by Nate Berkopec](https://www.speedshop.co/2017/12/04/malloc-doubles-ruby-memory.html), which should be considered as a tradeoff, as it may affect overall performance of memory allocation and GC cycles.
## Importance of Changes
When working on performance improvements, it's important to always ask yourself
executor running in privileged mode. If you're using the shared Runners on GitLab.com,
this is enabled by default.
Privileged mode is not necessary if you've [disabled Docker in Docker
for SAST](#disabling-docker-in-docker-for-sast)
CAUTION: **Caution:**
If you use your own Runners, make sure that the Docker version you have installed
is **not**`19.03.00`. See [troubleshooting information](#error-response-from-daemon-error-processing-tar-file-docker-tar-relocation-error) for details.
...
...
@@ -144,6 +147,21 @@ under your project's settings:
You can avoid the need for Docker in Docker by running the individual analyzers.
This does not require running the executor in privileged mode. For example:
```yaml
include:
template:SAST.gitlab-ci.yml
variables:
SAST_DISABLE_DIND:"true"
```
This will create individual `<analyzer-name>-sast` jobs for each analyzer that runs in your CI/CD pipeline.
### Overriding the SAST template
If you want to override the job definition (for example, change properties like
...
...
@@ -173,6 +191,7 @@ The following are Docker image-related variables.
| `SAST_ANALYZER_IMAGE_PREFIX` | Override the name of the Docker registry providing the default images (proxy). Read more about [customizing analyzers](analyzers.md). |
| `SAST_ANALYZER_IMAGE_TAG` | Override the Docker tag of the default images. Read more about [customizing analyzers](analyzers.md). |
| `SAST_DEFAULT_ANALYZERS` | Override the names of default images. Read more about [customizing analyzers](analyzers.md). |
| `SAST_DISABLE_DIND` | Disable Docker in Docker and run analyzers [individually](#disabling-docker-in-docker-for-sast). |
| `SAST_PULL_ANALYZER_IMAGES` | Pull the images from the Docker registry (set to 0 to disable). Read more about [customizing analyzers](analyzers.md). |
