- Aug 29, 2018
-
-
Thcipriani authored
-
Thcipriani authored
-
Thcipriani authored
-
- Aug 14, 2018
-
-
Dduvall authored
Change-Id: I91cac046976cf88d513487e27e1b52bf77a6f974
-
Dduvall authored
The builder configuration has proven useful for supporting generic pre-entrypoint commands such as dependency managers not otherwise supported by specific Blubber configuration. Adding additional `builder.requirements` config expands support for such commands by allowing the user to specify files that should be copied into the image before the builder command runs. To support this extra configuration, `builder` had to be changed from a simple string to a mapping. The builder command must now by given as `builder.command`. The pattern of creating parent directories, copying files, and executing one or more commands prior to the entrypoint has become a common one. Some of the implementation of this pattern was moved from `PythonConfig` into shared build macros `build.SortFilesByDir` and `build.SyncFiles`. All config types that must have requirements files copied over independently of the entire source tree (`PythonConfig`, `BuilderConfig`, `NodeConfig`) now delegate to these functions. Change-Id: I67f33034f22cee2851ec866cfb07ab20c23eba8c
-
- Aug 10, 2018
-
- Aug 09, 2018
-
-
Dduvall authored
The `lint` rule was previous surfacing purely whitespace output of `gofmt` as a failure and running `go tool vet` instead of `go vet` (the former accepts source files as arguments, the latter takes package names). A real lint error in `config/node.go` was also fixed. Change-Id: I7f13dcbb45a448f1ecac87e97527a7f4ed4d2c48
-
- Aug 02, 2018
-
-
Thcipriani authored
-
- Aug 01, 2018
-
-
Dduvall authored
With the move to Gerrit and away from Phabricator/Arcanist, we lost an easy way to run linters. New rules have been added to the `Makefile`, `lint`, `unit`, and `test` that run linters, unit tests, and both linters and unit tests, respectively. Bug: T200452 Change-Id: I0742daaa14389841d88f13eba47dee07ac127cf2
-
Hashar authored
Change-Id: I4b6a2ece0f0096c49ab4dba06f358e794941f94d
-
- Jul 31, 2018
-
-
Dduvall authored
All import paths and other references to the previously Blubber repo in Phabricator have been changed to point to the new Gerrit project URL. Note that building or testing against this change will require you to move your working directory to `$GOPATH/src/gerrit.wikimedia.org/r/blubber`. Contribution documentation was updated to reflect the changes, and the `.arcvendor` submodule has been removed. Bug: T200452 Change-Id: I3ab23e420b2825e86e2bd7c9b3da9d4de23edaa1
-
- Jul 26, 2018
-
-
Thcipriani authored
Summary: Ran into a Docker error when attempting to build a node project with more than one requirements files: When using COPY with more than one source file, the destination must be a directory and end with a / Reviewers: dduvall, #release-engineering-team Reviewed By: dduvall, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D1080
-
- May 31, 2018
-
-
Thcipriani authored
Reviewers: dduvall, #release-engineering-team Reviewed By: dduvall, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D1065
-
- May 25, 2018
-
-
Dduvall authored
-
Dduvall authored
-
Dduvall authored
Summary: Bump version to v0.4.0. Test Plan: Run `make && blubber --version` and verify the output. Reviewers: thcipriani, #release-engineering-team Reviewed By: thcipriani, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D1060
-
- May 24, 2018
-
-
Dduvall authored
Summary: See upstream bug https://github.com/golang/go/issues/24359 Test Plan: Run `make` from a symlink targetting the blubber directory under `GOPATH` and verify that `blubber --version` outputs the version and commit. Reviewers: thcipriani, #release-engineering-team Reviewed By: thcipriani, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D1061
-
Thcipriani authored
Reviewers: dduvall, #release-engineering-team Reviewed By: dduvall, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D1059
-
- May 20, 2018
-
-
Dduvall authored
-
Reviewers: thcipriani, dduvall, #release-engineering-team, mmodell Reviewed By: dduvall, #release-engineering-team, mmodell Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D1052
-
Dduvall authored
Summary: Change `node.dependencies` flag to `node.requirements`, the list of files (typically `package.json` and either `npm-shrinkwrap.json` or `package-lock.json`) that declare dependencies to be installed with NPM. Test Plan: Run `go test ./...`. Try it out with something like Mathoid. Reviewers: thcipriani, hashar, mobrovac, Joe, akosiaris, #release-engineering-team Reviewed By: thcipriani, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D1058
-
Reviewers: thcipriani, dduvall, #release-engineering-team Reviewed By: dduvall, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D1054
-
- May 19, 2018
-
-
Summary: We want to also install the wheels, not just generate them. To do that we add a pip install -t <target> command. For this to work however, we need to install into /opt/lib/python and amend the environment accordingly to set PYTHONPATH and PATH in order for the libraries and binaries to be accessible. Reviewers: dduvall, thcipriani, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D1057
-
- Apr 30, 2018
-
-
Dduvall authored
Summary: Ensure that `DEBIAN_FRONTEND=noninteractive` is set before injecting `apt-get install` instructions. Test Plan: Run unit tests. Reviewers: thcipriani, mmodell, hashar, demon, #release-engineering-team Reviewed By: thcipriani, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D1032
-
- Apr 05, 2018
-
-
Dduvall authored
Summary: Since `config.Validate` was changed to take any interface as an argument, many of the validation tests can be refactored in a way that avoids having to parse the full config context in YAML and instead validates each specific config struct directly. The new test pattern is simpler and less prone to future breakage should unrelated parts of the overall config change. Tests that rely on root config context were left unchanged. Test Plan: Run `go test ./...`. Reviewers: thcipriani, demon, hashar, mmodell, #release-engineering-team Reviewed By: thcipriani, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D1023
-
Dduvall authored
Summary: Introduced a `version` config field that must be specified and match `config.CurrentVersion`. Changed `config.ReadConfig` to use `yaml.UnmarshalStrict` to ensure that errors are surfaced when unknown/bad fields are present in the given YAML config. A smaller `config.VersionConfig` is now unmarshaled first to prevalidate the new `version` field before the entire config is parsed. Fixes T191460 Test Plan: Run `go test ./...`. Run `blubber` against some configuration containing invalid fields and ensure that it surfaces a YAML error. Reviewers: thcipriani, demon, hashar, mmodell, mobrovac, #release-engineering-team Reviewed By: thcipriani, #release-engineering-team Tags: #release-engineering-team Maniphest Tasks: T191460 Differential Revision: https://phabricator.wikimedia.org/D1021
-
- Apr 04, 2018
-
-
Dduvall authored
Git is used to set `meta.GitCommit` in `debian/rules`.
-
- Mar 28, 2018
-
-
Dduvall authored
Summary: To help new developers get on board, let's provide a `CONTRIBUTING.md` doc. Test Plan: Proofread with your eyeballs. Reviewers: thcipriani, demon, hashar, mmodell, zeljkofilipin, greg, Jrbranaa, #release-engineering-team Reviewed By: thcipriani, hashar, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D993
-
- Mar 23, 2018
-
-
Dduvall authored
-
- Mar 22, 2018
-
-
Dduvall authored
-
Dduvall authored
Summary: Bump the version to v0.3.0. Test Plan: Run `make && blubber --version` and verify the output. Reviewers: thcipriani, hashar, demon, #release-engineering-team Reviewed By: thcipriani, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D1014
-
Thcipriani authored
Test Plan: Run: blubber -p policy.example.yaml blubber.example.yaml production Be happy it works. Reviewers: dduvall, #release-engineering-team Reviewed By: dduvall, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D1013
-
Thcipriani authored
Summary: It occurred to me while looking at code in another patch that there is no need to have this function be public. Test Plan: `go test ./...` Reviewers: dduvall, #release-engineering-team Reviewed By: dduvall, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D1012
-
Dduvall authored
Summary: Use cases involving running of test suites and doc generation require more liberal ownership and read/write permission to application files. When `runs.insecurely` is set to `true`, the effective runtime user will be `lives.as`, the same user that owns the application files and installed dependencies. D999 is a complement to this change to allow restrictions on this and other potentially sensitive configuration. Depends on D999, D1002 Test Plan: Run `go test ./...`. Reviewers: thcipriani, mmodell, hashar, #release-engineering-team, demon Reviewed By: thcipriani, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D1003
-
Dduvall authored
Summary: The implementation of D984 did not include enforcing ownership for `build.CopyFrom` instruction and so artifacts copied from one image to another via `copies:` were problematically owned as root. In order to fix this behavior: 1. `config.ArtifactConfig` `build.CopyFrom` instructions are now injected duration `build.PhaseInstall` 2. `config.VariantConfig` calls `build.ApplyUser` for these artifact instructions as well using the `runs.as` user 3. `build.CopyAs` was refactored to wrap any `build.Instruction` which should only really be used with `build.Copy` or `build.CopyFrom`. Test Plan: Run `go test ./...`. Run `blubber` against configuration with a variant that uses `copies` and verify that the `COPY --from` instructions also include a `--chown` flag. Reviewers: thcipriani, mmodell, hashar, #release-engineering-team, demon Reviewed By: thcipriani, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D1002
-
- Mar 19, 2018
-
-
Dduvall authored
Summary: Implements a rough interface for validating configuration against arbitrary policy rules. Policies are provided as YAML and passed via the command line as file paths or remote URIs. The format of policies is: enforcements: - path: <path> rule: <rule> Where `<path>` is a YAML-ish path to a config field and `<rule>` is any expression our config validator understands (expressions built in by the validator library and custom tags defined in `config.validation.go`). Example policy: enforcements: - path: variants.production.base rule: oneof=debian:jessie debian:stretch - path: variants.production.runs.as rule: ne=foo - path: variants.production.node.dependencies rule: isfalse Command flag parsing was implemented in `main.go` to support the new `--policy=uri` flag and improve existing handling of `--version` and the usage statement. Test Plan: Run `go test ./...`. Reviewers: thcipriani, demon, hashar, mmodell, #release-engineering-team Reviewed By: thcipriani, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D999
-
- Mar 06, 2018
-
-
Dduvall authored
Summary: A new root and variant `python` config field is provided with two new fields below, `version` and `requirements`. The former, `version`, should specify the Python executable to use when executing related package installation commands and ostensibly the same executable that will be used to run the application. The latter, `requirements`, should specify all pip requirements files such that a compiler that supports layered filesystems (e.g. Docker) can output separate instructions that will invalidate cache layers for changes to those files independently of changes to the rest of the codebase. Python related instructions will be generated only if either `version` or `requirements` are given. Fixes T186545 Test Plan: Run `go test ./...`. Reviewers: thcipriani, hashar, demon, #release-engineering-team Reviewed By: thcipriani, #release-engineering-team Tags: #release-engineering-team Maniphest Tasks: T186545 Differential Revision: https://phabricator.wikimedia.org/D976
-
Thcipriani authored
Summary: Since we now use ldflags to set global variables it's helpful to automate via Make. Using a single global GOPATH seems to jive more with the best-practice established in the go community[0] and make testing/code review easier. [0]. <https://github.com/golang/go/wiki/GOPATH#use-a-single-gopath> Test Plan: run make Reviewers: dduvall, #release-engineering-team Reviewed By: dduvall, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D998
-
Dduvall authored
Summary: Introduce new `build.EntryPoint` and `build.WorkingDirectory` instructions to allow configuration to inject them instead of hard coding their generation in the Docker compiler. Simplified the Docker compiler to simply iterate over build phases as returned by a new function `build.Phases()`. Depends on D990 Test Plan: Run `go test ./...`. Reviewers: thcipriani, demon, hashar, #release-engineering-team Reviewed By: thcipriani, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D991
-
Dduvall authored
Summary: After adding a number of new instructions, it seemed there was a lot of redundant implementation between concrete types. This refactor uses just one concrete unexported type and a few options for determining the output format instead of one struct type per possible Docker instruction. The exported `docker.Instruction` interface and signature of `docker.NewInstruction` is unchanged with this refactor. Unit tests for Docker instruction compilation were simplified to not make type assertions but were otherwise left to ensure no regression. Depends on D984 Test Plan: Run `go test ./...`. Reviewers: thcipriani, demon, hashar, mmodell, #release-engineering-team Reviewed By: thcipriani, #release-engineering-team Tags: #release-engineering-team Differential Revision: https://phabricator.wikimedia.org/D990
-