1THIS IS OUT OF DATE
2===================
3
4This really hasn't been updated for awhile. Sorry about that.
5
6To use `copy-gold`:
7
81. Do a full KAT run:
9 1. Get a cluster.
10 2. Set up `DEV_KUBECONFIG` and `DEV_REGISTRY` to access your cluster.
11 3. `make push pytest KAT_RUN_MODE=envoy`
12 4. **MAKE SURE YOU GET A CLEAN TEST RUN.** This is super-critical.
13
142. Run `copy-gold /tmp/gold` to copy the snapshot directories of all the Ambassadors running in KAT back to your system in the `/tmp/gold` directory.
15
163. Compare `/tmp/gold` to `python/tests/gold` and see what, if any, changes have happened.
17 1. New directories should appear **only** if you've added tests.
18 2. Old directories should disappear **only** if you've removed tests.
19 3. Changes to `aconf.json` or `ir.json` files should happen **only** if you've changed test inputs.
20 4. Changes to `envoy.json` files should **really** happen **only** if you've changed things.
21
224. If there are **any** unexplained changes, **STOP**. Figure out what's going on, fix it, and rerun the steps above.
23
245. Once there are **no** unintended changes, copy everything from `/tmp/gold` to `python/tests/gold`.
25
26Original Notes
27--------------
28
29*Note*: `flynn/dev/watt` has the watt binary integrated into `entrypoint.sh` and such, so if
30you build Docker images from there you'll get watt running too. The 'copy snapshots and use them
31to mock stuff' bit is still very very relevant.
32
33*Note*: There's also the `test-dump.py` program that basically acts like `diagd`, but just
34works on a snapshot file directly and spits out AConf, IR, and V2 JSON files.
35
36
37Some notes on the dev loop I've been using working on watt integration:
38
391. Fire up a Kubernetes cluster (I used minikube for this, actually). Load it up with an Ambassador
40 (in this case, one that I built, but basically 0.52.1). `kubectl cp` the binary for the new discovery
41 thing - `watt` - in place.
42
43 `watt` basically works like the existing `kubewatch`: it constructs snapshots of relevant resource
44 sets and gives Ambassador a way to grab the snapshots. This means that I can
45
462. Register some services with Consul, grab a snapshot from `watt`, and save it to disk. Repeat a few
47 times, tweaking various things, so that I have a few sets of snapshots.
48
493. `kubectl cp` my snapshots back to my laptop (a Mac).
50
51Now for the fun(?) bit: what do these things really look like? How do the data structures act as I
52tweak things? How can I manipulate them into the internal Ambassador-native resources that I actually
53need? I could do this by constantly rebuilding Ambassador, pushing Docker images, and updating my
54Kube cluster, but that’s _way_ too slow for the inner loop. Instead:
55
564. Fire up a faked version of `watt` that’s literally a Flask app serving one of my snapshot files,
57 then run `diagd` by hand.
58
595. `diagd` is the (increasingly-misnamed) piece of Ambassador that basically embodies the compiler
60 part of Ambassador: it takes these snapshots and generates the Envoy config. Running it by hand
61 locally means that I can feed it snapshots from my fake `watt` and watch what it does, then make
62 a change and try again.
63
64My inner development loop is now “make a change in some Python code, hit up-arrow RETURN in two
65shell windows, see if things make sense.” This is pretty quick. Once I think it’s working correctly,
66yeah, _then_ I have to build images and `kubectl apply`, but my first test cluster is `minikube`
67with three pods in it, so that’s pretty quick.
68
69So that’s how we do things at Datawire: figure out how to isolate pieces so that we _don’t_ have to
70suffer with bringing clusters up and down, etc., until we’re ready to let CI have its evil way with
71a new feature. It’s not nearly as smooth as we’d like for Ambassador, but it’s getting better. :)
View as plain text