This PR should meet the following requirements from #8

• [x] Use @ syntax for specifying version (e.g. https://github.com/hashicorp/[email protected])
• [x] Add pack or registry @tag
• [x] Add pack or registry @release
• [x] Add pack or registry @SHA
• [x] Add or update pack or registry @latest - will store in the global cache and log SHA at time of installation
• [x] Add registry with alias
• [x] List registries and packs at all versions

The following commands all yield an empty jenkins.nomad

nomad-pack render jenkins -f jenkins/vars.nomad > jenkins.nomad nomad-pack render jenkins -f jenkins/vars.nomad &> jenkins.nomad nomad-pack render jenkins -f jenkins/vars.nomad 2>&1 | tee jenkins.noma nomad-pack render jenkins -f jenkins/vars.nomad | tee jenkins.nomad nomad-pack render jenkins -f jenkins/vars.nomad |& tee jenkins.nomad

I get this error when I try to plan a pack. It works properly when rendering and when running though.

➜  pack plan ./packs/promtail

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x1b422ab]

goroutine 1 [running]:
github.com/hashicorp/nomad-pack/internal/runner/job.formatTaskGroupDiff({0x0, 0xc0006908a0, 0x0, 0xc00009e618, 0xc000690890, 0x0}, 0x2, 0x0, {0x2036ed8, 0xc00054c8a0})
	/Users/mike/Code/nomad-pack/internal/runner/job/format.go:105 +0x3ab
github.com/hashicorp/nomad-pack/internal/runner/job.formatJobDiff({0xc00009e480, 0xc0006905f0, 0x0, 0xc00009e528, 0xc0006905e0}, 0x0, {0x2036ed8, 0xc00054c8a0})
	/Users/mike/Code/nomad-pack/internal/runner/job/format.go:90 +0x5a5
github.com/hashicorp/nomad-pack/internal/runner/job.(*Runner).outputPlannedJob(0xc00091fb08, {0x2036ed8, 0xc00054c8a0}, 0xc00022ea20, 0xc0004e7a80)
	/Users/mike/Code/nomad-pack/internal/runner/job/plan.go:118 +0xab
github.com/hashicorp/nomad-pack/internal/runner/job.(*Runner).PlanDeployment(0xc000540190, {0x2036ed8, 0xc00054c8a0}, 0xc00009e498)
	/Users/mike/Code/nomad-pack/internal/runner/job/plan.go:57 +0x627
github.com/hashicorp/nomad-pack/cli.(*PlanCommand).Run(0xc00009e1b0, {0xc0000300e0, 0x0, 0x0})
	/Users/mike/Code/nomad-pack/cli/plan.go:104 +0x659
github.com/mitchellh/cli.(*CLI).Run(0xc00012e140)
	/Users/mike/go/pkg/mod/github.com/mitchellh/[email protected]/cli.go:262 +0x5f8
github.com/hashicorp/nomad-pack/cli.Main({0xc0000300c0, 0x3, 0x3})
	/Users/mike/Code/nomad-pack/cli/main.go:94 +0x2ad
main.main()
	/Users/mike/Code/nomad-pack/main.go:12 +0x7f

Switches implementation to use URL parsing to enable hostnames other than github.com. This fixes #161.

There is an open question about unparsable URLs in the Pack.URL field of the metadata. For now, I opted to output the error message in the list output but continue rather than failing completely, since the pack would still be able to be rendered.

the newly added jenkins pack contains an error and when i try to run it the error returned by nomad-pack is less than useful. I had to render the pack and manually and run the job, to get presented with some reasonable feedback from the system.

to reproduce.

packs ❯ pwd nomad-pack-community-registry/packs packs ❯ nomad-pack run jenkins -f jenkins/vars.nomad ! Failed To Parse Job Specification

Error:   400 Bad Request
Type:    client.GenericOpenAPIError
Context:
         - Template Name: jenkins/templates/jenkins.nomad.tpl
         - Registry Name: dev
         - Pack Name: jenkins
         - Pack Ref: dev
         - Deployment Name: jenkins

packs ❯ nomad-pack render jenkins -f jenkins/vars.nomad

safe the output to a file (see #163 for my bug report about not being able to pass directly to file) and run it using the nomad cli

packs ❯ nomad run jenkins.nomad
Error getting job struct: Failed to parse using HCL 2. Use the HCL 1 parser with nomad run -hcl1, or address the following issues: jenkins.nomad:61,7-13: Unsupported argument; An argument named "config" is not expected here. Did you mean to define a block of type "config"?

Description

make bootstrap is a necessary step for development that needs to run at least once.

Reminders

• [ ] Add CHANGELOG.md entry (not necessary)

• Implement --render-to-file
• Add non-interactive ErrorWithContext
• Format Error Context Elements
• Add auto-approve flag
• Update to --to-dir; adopt --auto-approve flag for overwrite
• Implemented Input in glintUI
• Missed the callsite renaming confirmOverwrite

Closes #109

Summary

Some packs does (may) not have a namespace variable and I tried to deploy to a different namespace using env var NOMAD_NAMESPACE but this didn't work.

We would like to see internal registry where we have deployed our nomad cluster and and we can deploy registry like container registry for eg. Harbor or we can try to integrate to Harbor registry

When I tried switching the upstream repo URL while running nomad-pack init the --from flag appears to have no effect:

nomad-pack on  main via 🐹 v1.16.5 
❯ ./bin/nomad-pack init --from https://github.com/hashicorp/nomad-pack-registry.git
  created global cache directory: /Users/lennon/.nomad/packs
  Initializing registry...
  Downloading source from [email protected]:hashicorp/nomad-pack-registry.git
  Installing into: /Users/lennon/.nomad/packs/default

The local clone created confirms the the upstream is using the SSH ([email protected]:...) URL, despite the --from line specifying an HTTP origin:

❯ cat ~/.nomad/packs/default/.git/config 
[core]
	repositoryformatversion = 0
	filemode = true
	bare = false
	logallrefupdates = true
	ignorecase = true
	precomposeunicode = true
[remote "origin"]
	url = ssh://[email protected]/hashicorp/nomad-pack-registry.git
	fetch = +refs/heads/*:refs/remotes/origin/*
[branch "main"]
	remote = origin
	merge = refs/heads/main

This is on main as of a7ef2357

I was working on Nomad Pack deployments and found that they couldn't submit jobs correctly with max_client_disconnect. From what I can tell this is related to the Nomad libraries that are used (I am using tech preview 2). Can you please pull in the PR #249 (once they update it to 1.3.1 😄 ) as I believe that will solve it. Thank you @mikenomitch !

I am trying to build a job that contains the following network and service resources and i currently have to specify the port number in both. Of course i can turn this into a pack variable and be done with it, but that feels weird/wrong since the port number is incredibly static in this particular case.

network {
  mode = "bridge"
  port "mysql" {
    to = 3306
  }
}

[[ if .my.register_consul_service ]]
service {
  name = "[[ .my.consul_service_name ]]"
  tags = [[ .my.consul_service_tags | toStringList ]]
  port = "mysql"
  connect {
    sidecar_service {
      tags = [""]
      proxy {
        local_service_port = 3306
      }
    }
  }
}
[[ end ]]

In terraform we have something called locals, this would be a nice solution to be able to use this with packs as well. While attempting to use a local block in the variables.hcl file nomad-pack fails with the following error

nomad-pack run packs/mysql -f vars/mysql.hcl -f vars/lab.hcl
! Failed To Process Pack

        Error:   Blocks of type "locals" are not expected here.
        Type:    *errors.errorString
        Context: 
                 - HCL Range:
/Users/attachmentgenie/DevShed/projects/attachmentgenie/nomad-pack-attachmentgenie-registry/pac
                 - Registry Name: <<local folder>>
                 - Pack Name: mysql
                 - Pack Ref: <<none>>
                 - Pack Path:
/Users/attachmentgenie/DevShed/projects/attachmentgenie/nomad-pack-attachmentgenie-registry/pac
                 - Deployment Name: mysql

When trying to use nomad-pack generate pack, I initially ran the command bare (mostly by accident).

I'd usually expect that either the command would interactively ask for what it wanted, or to fail with usage information that would guide me to the correct invocation.

However instead it failed with:

! Error Parsing Args Or Flags

        Error:   this command requires exactly 1 args(s), received 0
        Type:    *errors.errorString

This is somewhat a jarring error string for incorrect input (and feels more like what I'd expect if some other, less expected, failure occurred).

Hey,

I was following the tutorial in the hashcorp website, and when i was doing this command gave me this error

nomad-pack info .

panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x18c91c1] goroutine 1 [running]: github.com/hashicorp/nomad-pack/internal/pkg/variable.(*Parser).loadPackFile(0xc0004fce40, 0x0, 0xc0005bf730, 0x203000, 0x203000, 0x187, 0xf) /home/circleci/project/project/internal/pkg/variable/parser.go:174 +0xa1 github.com/hashicorp/nomad-pack/internal/pkg/variable.(*Parser).parseRootFiles(0xc0004fce40, 0xc0005bf820, 0x1862b74, 0x1ffade0) /home/circleci/project/project/internal/pkg/variable/root.go:14 +0x125 github.com/hashicorp/nomad-pack/internal/pkg/variable.(*Parser).Parse(0xc0004fce40, 0xc0004fce40, 0x0, 0x0, 0x0) /home/circleci/project/project/internal/pkg/variable/parser.go:94 +0x45 github.com/hashicorp/nomad-pack/internal/cli.(*InfoCommand).Run(0xc000060360, 0xc00008c050, 0x1, 0x1, 0xc000060200) /home/circleci/project/project/internal/cli/info.go:60 +0x3a5 github.com/mitchellh/cli.(*CLI).Run(0xc00054a8c0, 0xc0005bfef8, 0xc00000e008, 0x1) /go/pkg/mod/github.com/mitchellh/[email protected]/cli.go:262 +0x41a github.com/hashicorp/nomad-pack/internal/cli.Main(0xc00008c030, 0x3, 0x3, 0x0) /home/circleci/project/project/internal/cli/main.go:93 +0x225 main.main() /home/circleci/project/project/main.go:12 +0xa5

Not that Nomad-Pack needs to be like Helm, but for work I came across a use-case, namely how can I take a community Helm Chart and make changes to it without forking / committing to the repo, and was left wondering "how would I do that with Nomad-Pack"?

It looks like there would be a way to perhaps do a render and then run a script against that output. Or perhaps some hcl-to-json converstion before processing. Either way, it would be cool to see some docs walking through how one would alter or add to an existing community pack.

Thanks, Sam

hello we're trying to develop our nomad pack,

as our nomad-pack is extended version of nomad-autoscaler community pack

after that want to run the pack with custom variables, notes that the variables.tf, metadata.hcl is no different from nomad-autoscaler community pack we just adding vault stanza in the config for now.

then we run the command like below :

nomad-pack run ./packs/nomad-autoscaler -f ./config/development/development_variables.hcl

but got many error :

❯ nomad-pack run ./packs/nomad-autoscaler -f ./config/development/development_variables.hcl 
! Failed To Process Pack

        Error:   There is no variable named "job_name". An override file can only override a variable that was already declared in a primary configuration file.
        Type:    *errors.errorString
        Context: 
                 - HCL Range: ./config/development/development_variables.hcl:1,1-33
                 - Registry Name: <<local folder>>
                 - Pack Name: nomad_autoscaler
                 - Pack Ref: <<none>>
                 - Pack Path: /home/kholis/packs/nomad-autoscaler
                 - Deployment Name: nomad-autoscaler

! Failed To Process Pack

        Error:   There is no variable named "datacenters". An override file can only override a variable that was already declared in a primary configuration file.
        Type:    *errors.errorString
        Context: 
                 - HCL Range: ./config/development/development_variables.hcl:2,1-28
                 - Registry Name: <<local folder>>
                 - Pack Name: nomad_autoscaler
                 - Pack Ref: <<none>>
                 - Pack Path: /home/kholis/packs/nomad-autoscaler
                 - Deployment Name: nomad-autoscaler
                 
...

this is inside development_variables.hcl

job_name    = "nomad_autoscaler"
datacenters = ["us-west-2"]
region      = "global"
namespace   = "default"

autoscaler_agent_network = {
  autoscaler_http_port_label = "http"
}
autoscaler_agent_task = {
  driver               = "docker",
  version              = "0.3.7",
  additional_cli_args  = ["-nomad-address=http://$${attr.unique.network.ip-address}:4646", "-http-bind-address=0.0.0.0"],
  config_files         = ["./config/development/development_config.hcl"],
  scaling_policy_files = ["./config/development/development_policy.hcl"],
}

but when we try to use l use nomad-pack with the default registry

❯ nomad-pack run nomad_autoscaler -f ./config/development/development_variables.hcl
  Evaluation ID: 37f4ce41-411b-df92-41ca-2f4d55f7744b
  Job 'nomad_autoscaler' in pack deployment 'nomad_autoscaler@latest' registered successfully
Pack successfully deployed. Use nomad_autoscaler with --ref=latest to manage this deployed instance with plan, stop, destroy, or info

its success.

is this intended behavior or bug?

I stumbled upon it in the Packer documentation and assumed this was an HCL2 feature available across all products.

It doesn't look like nomad-pack supports it making variable validation somewhat painful. The best I could find was to have my whole template surrounded by a giant [[ if ]] [[ end ]] condition.

Sample variable:

variable "location_type" {
  type        = string
  default     = "dc"
  description = "Type of physical location (dc, pop or stx)."

  validation {
    condition     = contains(["dc", "pop", "stx"], var.location_type)
    error_message = "No bueno"
  }
}

Output I'm getting:

! Failed To Process Pack

        Error:   Blocks of type "validation" are not expected here.
        Type:    *errors.errorString