Juju is a model-driven Operator Lifecycle Manager (OLM).

Branch 3 in the service to application rename adventure. Here we rename state methods and other artefacts, plus some testing factory methods. As a driveby, delete some old uniter upgrade steps.

(Review request: http://reviews.vapour.ws/r/4955/)

There are 3 main changes in this pr:

1. Added a cleanup metrics and delete functions to state to allow metrics to be deleted
2. Added an API endpoint for calling the CleanupMetrics function
3. Added a worker that will periodically call the cleanupmetric function

The worker never gets called - except in tests - that will come in a follow up pr. The CleanupMetrics function currently just returns an error. In a follow up it will return a list of the uuids that were deleted.

There was a report of a race in statushistorypruner tests https://bugs.launchpad.net/juju-core/+bug/1492095 this removes the race which consisted in a variable being written/read in various routines.

(Review request: http://reviews.vapour.ws/r/2592/)

Description of change

The following changes validates the LXDProfile of a charm and bundle if it's from the charm store.

The following checks that the profile is correctly validated when deploying.

Notes

Feedback for where the validateXXX profile function should be, it's currently duplicated and would make sense to have it somewhere for reuse.

QA steps

Ensure that you have the feature flag turned on when bootstrapping, as this needs to be set.

export JUJU_DEV_FEATURE_FLAGS=lxd-profile

The cases I checked are the following, I'm sure I probably missed one, so if that's the case let me know so I can ensure that we cover those cases as well.

juju deploy ./testcharms/charm-repo/bundle/lxd-profile/bundle.yaml
juju deploy ./testcharms/charm-repo/bundle/lxd-profile-fail/bundle.yaml
juju deploy cs:~juju-qa/bionic/lxd-profile-0
juju deploy cs:~juju-qa/bionic/lxd-profile-fail-0

Documentation changes

When deploying a bundle with a lxd-profile configuration that isn't valid, it should show an error to the user explaining what went wrong and potentially how to fix it.

Bug reference

TBA

Description of change

This change add the support of OpenSUSE Leap (42 series) to Juju.

Main reason for the change is that most of our NVF run on top of OpenSUSE/SLES and I was doing some hands-on for evaluating Juju as a generic VNF-M. This PR (and other for juju/utils) is the result of my work.

With these PR, Juju users are able to deploy OpenSUSE charms in LXD and in manual provisioned OpenSUSE host.

This PR depends on: https://github.com/juju/utils/pull/277

QA steps

A part of the testing that I added, it is possible to deploy an OpenSUSE charm: https://github.com/marcmolla/juju-OpenSUSE/tree/master/charms/test-opensuseleap

using a modified OpenSUSE image: https://github.com/marcmolla/juju-OpenSUSE/blob/master/lxd-opensuse-42.2-image.md

I also describe how I tested the software locally: https://github.com/marcmolla/juju-OpenSUSE/blob/master/test-compiled-Juju.md

For all those testing we also need another PR to juju/utils (https://github.com/juju/utils/pull/272)

Documentation changes

If this PR is accepted, we should include reference to OpenSUSE in general doc.

Bug reference

N/A

Description of change

Make proxy settings global - previously the settings were only used on 'ubuntu' user login shell, with this change the settings are global for all users using /etc/profile.d and, on systems using systemd, also set for services launched using systemd via /etc/systemd/{system,user}.conf.d/juju-proxy.conf.

QA steps

1. Run unit tests
2. Verify that if http-proxy option is set it's being set in the environment - e.g. services running via systemd should have proper env variables set.

Documentation changes

This change has to be noted in release notes and docs.

Bug reference

https://bugs.launchpad.net/juju/+bug/1666353

When machine addresses are updated, a new server certificate is made so that connections over those addresses can be made securely. Instead of restarting the API server to use the new certificate, the net listener is modified to replace the current certificate with the new one without stopping and starting the server. While the certificate is being updated, new connections to the server are disallowed.

(Review request: http://reviews.vapour.ws/r/667/)

For collections that contain data for multiple environments, getCollection now returns a wrapped collection that automatically filters by environment UUID as required. This reduces the risk of unintended data leakage between environments.

There are some areas where an actual mgo.Collection is required. These now use the new State.getRawCollection method.

Now that this is handled automatically, this change also removes several TODOs regarding filtering by environment UUID.

A later branch will remove all the now-unnecessary docID calls that currently exist in state.

(Review request: http://reviews.vapour.ws/r/551/)

Now tests which involve calls to the system and expect a "not found" response error are platform-agnostic.

This pull depends on the platform-specific regex-es from the utils package, which in turn depends on the addition of the NoSuchUser error type in the errors package.

(Review request: http://reviews.vapour.ws/r/548/)

This should fix https://bugs.launchpad.net/juju-core/+bug/1654528 by making juju use set of TLS cipher suites that are compatible with the stock GnuTLS suites provided by precise and trusty.

Note that this branch is targeted at 1.25 and the utils dependency is in the 1.25 branch of that repo.

follow up to #6580

@perrito666 @babbageclunk

This removes uvtool as a dependency, which should allow kvm containers to work on arm64, amd64, and ppc64el. QA has only been done by me on amd64 with vmaas.

NB: Since merging develop back into this branch there are some network updates that break kvm containers. When the rest of the network updates are complete and land it should be fine again. In order to test, once you have checked out the branch you will need to undo the changes so you can QA. Do that with git revert -m 1 37a1a3b87fc54dc7391b4027e385ac48722ddf42 --no-commit

QA:

1. juju bootstrap vmaas21 kvm/purego --build-agent
2. juju add-machine
3. juju add-machine --series trusty
4. juju deploy ubuntu --to kvm:0
5. juju deploy mysql --to kvm:0
6. juju deploy wordpress --to kvm:1
7. juju deploy postgresql --to kvm:1 --series xenial
8. juju add-relation wordpress mysql

verify that machines 0 and 1 came up with xenial and trusty respectively verify that machines 0/kvm/0 cam up with xenial and 0/kvm/1 with trusty verify that machine 1/kvm/0 came up with trusty and 1/kvm/1 with xenial verify the relation was added... why not create a wordpress blog to be sure.

9. juju remove-application ubuntu
10. juju remove-application postgresql

verify that the disk images for those are gone ,by juju ssh 0 (or 1) and look in /var/lib/juju/kvm/guests to be sure they were removed.

The following attempt to speed up github/generate workflow. The go generate command takes forever to trawl the codebase, so instead, we'll use grep to select the go generates and do it manually.

We then have the option of parallelizing the generate commands.

Checklist

• [x] Comments saying why design decisions were made

QA steps

See the github action.

The following allows repl access to all substraits. Currently this only works on LXD, opening it up to substrait is advantageous.

Checklist

• [x] Comments saying why design decisions were made

QA steps

$ make go-install
$ juju bootstrap aws test
$ make juju-dqlite-repl

Bug reference

https://warthogs.atlassian.net/browse/JUJU-2353

Forward merge of 2.9 into 3.0

Conflicts: api/controller/caasapplicationprovisioner/client.go apiserver/backup_test.go apiserver/facades/client/backups/create.go apiserver/facades/controller/caasapplicationprovisioner/mock_test.go caas/kubernetes/provider/application/application.go caas/mocks/application_mock.go state/backups/db.go worker/caasapplicationprovisioner/application_test.go

This set of changes allow bootstrapping to a LXD vm instead of a container. Most of the work was already done, but it wasn't selecting the right images to bootstrap from. Additionally, there are some issues attempting to find the running status correctly for a VM. The simple solution is to just implement a retry mechanism, with enough back off, but there might be a better approach to this.

The tests don't currently pass, but future commits will fix those.

A simple QA to get up and running is the following:

$ juju bootstrap lxd --constraints virt-type=virtual-machine

We store a different alias for the VM image independently from a container image, as they're different. LXD itself has an API for remote lookup that doesn't require this to be explicit, but we do for the alias. This means that we suffix all aliases with /vm to indicate the difference. For containers, we leave as is, so that operators can reuse the existing deployments without a new fetch.

Note: I'm not sold on the full "vitural-machine" and "container" naming, but I'm just passing through what LXD expects.

Checklist

If an item is not applicable, use ~strikethrough~.

• [x] Code style: imports ordered, good names, simple structure, etc
• [x] Comments saying why design decisions were made
• [x] Go unit tests, with comments saying what you're testing
• [ ] Integration tests, with comments saying what you're testing
• [x] doc.go added or updated in changed packages

QA steps

$ juju bootstrap lxd --constraints virt-type=virtual-machine

Documentation changes

@tmihoc This will now deploy to LXD vms via a virt-type constraint.

Bug reference

https://warthogs.atlassian.net/browse/JUJU-2351

Remove the CharmStore macaroons as a side effort to retiring CharmStore from the client entirely.

Relates to PRs such as #14968, #14985.

https://warthogs.atlassian.net/browse/JUJU-2230 -- removing the CharmStore entirely from the client is coming up next.

QA Steps

As CharmStore charms are not supported, this doesn't change any behavior.

This is the start of threading through context.Context from the rpc methods through to the actual method calling. Currently context.Context is optional from the rpcreflect library, yet, it seems like a missed opportunity to not use this. With the new database changes, we would want to be able to cancel a sql transaction. Additionally, we could use the context.Context for opentracing to help debug/diagnose running issues.

The code makes a lot of changes, but it's all very mechanical.

Why this change is needed and what it does.

Checklist

If an item is not applicable, use ~strikethrough~.

• [ ] Code style: imports ordered, good names, simple structure, etc
• [ ] Comments saying why design decisions were made
• [ ] Go unit tests, with comments saying what you're testing
• [ ] Integration tests, with comments saying what you're testing
• [ ] doc.go added or updated in changed packages

QA steps

Commands to run to verify that the change works.

QA steps here

Documentation changes

How it affects user workflow (CLI or API). Delete section if not applicable.

Bug reference

Link to Launchpad bug. Delete section if not applicable.