A Cloud Native Buildpack that provides the Open Liberty runtime

Last update: Dec 21, 2022

Comments: 15

`gcr.io/paketo-buildpacks/open-liberty`

The Paketo Open Liberty Buildpack is a Cloud Native Buildpack that contributes Open Liberty for Java EE support.

Behavior

This buildpack will participate all the following conditions are met

(
  
   
    /WEB-INF exists OR
  
    
     /server.xml exists OR
  
     
      /wlp/usr/servers/*/server.xml exists
) AND (
  Main-Class is NOT defined in the mainfest
)

The buildpack will do the following:

Requests that a JRE be installed
Contribute an Open Liberty runtime to wlp.install.dir
Create a server called defaultServer
Contributes web process type
At launch time, symlinks to ${wlp.install.dir}/usr/servers/defaultServer/dropins/.

The buildpack will support all available profiles of the two most recent versions of the Open Liberty runtime. Note that because the Open Liberty versioning scheme is not conformant semantic versioning, an Open Liberty version like 21.0.0.11 is defined here as 21.0.11, and should be referenced as such.

Configuration

Environment Variable	Description
`$BP_OPENLIBERTY_VERSION`	The version of Open Liberty to install. Defaults to the latest version of the runtime.
`$BP_OPENLIBERTY_PROFILE`	The Open Liberty profile to use. Defaults to `full`.
`$BP_OPENLIBERTY_APP_NAME`	If the server.xml does not specify a context root, Open Liberty will use this value as the context root. Defaults to the value of `$CNB_APP_DIR`, currently `workspace`.
`$BP_OPENLIBERTY_LOG_LEVEL`	Sets the logging level. If not set, attempts to get the buildpack's log level. If unable, defaults to `INFO`

Default Configurations that Vary from Open Liberty's Default

By default, the Open Liberty buildpack will log in json format. This will aid in log ingestion. Due to design decisions from the Open Liberty team, setting this format to any other value will prevent all log types from being sent to stdout and will instead go to messages.log. In addition, the log sources that will go to stdout are message,trace,accessLog,ffdc,audit.

All of these defaults can be overriden by setting the appropriate properties found in Open Liberty's documentation. They can be set as environment variables, or in bootstrap.properties.

Bindings

The Open Liberty buildpack will accept a single binding of type open-liberty. There are currently two files supported in the binding:

server.xml. If specified, this file will replace the defaultServer's server.xml and is not subject to any post-processing; therefore, any variable references therein must be resolvable.
bootstrap.properties. If specified, this file will replace the defaultServer's bootstrap.properties. This is one place to define variables used by server.xml.

Both files are optional, as is the binding itself.

License

This buildpack is released under version 2.0 of the Apache License.

Owner

Paketo Buildpacks

Cloud Native Buildpack implementations for popular programming language ecosystems

https://github.com/paketo-buildpacks/open-liberty

Comments

Include features and ifixes in SBoM
Summary

Fixes #137.

This change scans the base and distribution layers to include dependencies in the SBoM. I could not figure out a more targeted approach since there is no way to query the Liberty feature installer tool to see which jars and versions it installs for a given feature at the moment.

I also tried investigating on providing the information in a manner useful to vulnerability scanners like grype but it did not look possible since Liberty only includes the version of Liberty that a vulnerability is resolved in and does not include the which feature is vulnerable and what version of the feature it is resolved in. I talked with the lead maintainer for Open Liberty and he was open to improving this process.

I was not sure the best approach for including feature and iFix information in the SBoM. If there is a better approach to include useful information, please let me know and I can try to update.

Use Cases

Include feature and iFix information in the SBoM.

Checklist

[X] I have viewed, signed, and submitted the Contributor License Agreement.

[X] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)

[X] I have added an integration test, if necessary.

[X] I have reviewed the styleguide for guidance on my code quality.

[X] I'm happy with the commit history on this PR (I have rebased/squashed as needed).
Documentation is not clear
The configuration information is not clear enough

$BP_LIBERTY_VERSION

need an example of how the version looks like, 22.0.0.7

what value for the latest version?

$BP_LIBERTY_PROFILE

beside full, what are the other values?

$BP_LIBERTY_CONTEXT_ROOT

if the server.xml has an application named app defined, will also generate and override the context root?

$BP_LIBERTY_FEATURES

better point to a Open Liberty features doc, e.g. https://openliberty.io/docs/latest/reference/feature/feature-overview.html

$BP_LIBERTY_EXT_CONF_SHA256

how the values look like? list the values or point to another doc

The image built by paketo with kernel profile does not work

the server.xml contains cdi-3.0, jsonb-2.0, jsonp-2.0, restfulWS-3.0 features

built the image by the following command

pack build --path ./system ^
--env BP_JAVA_APP_SERVER=liberty ^
--env BP_LIBERTY_FEATURES="cdi-3.0 jsonb-2.0 jsonp-2.0 restfulWS-3.0" ^
--env BP_MAVEN_BUILT_ARTIFACT="target/*.[ejw]ar src/main/liberty/config/*" ^
--env BP_LIBERTY_PROFILE=kernel ^
--buildpack paketo-buildpacks/eclipse-openj9 ^
--buildpack paketo-buildpacks/java system:1.0-SNAPSHOT

ran the image and got following error

CWWKF0033E: The singleton features io.openliberty.servlet.api-5.0 and io.openliberty.servlet.api-3.0 cannot be loaded at the same time.  The configured features restfulWS-3.0 and restfulWS-3.0 include one or more features that cause the conflict. Your configuration is not supported; update server.xml to remove incompatible features.

Unable to use custom server.xml when building the image
What happened?

What were you attempting to do? Tried to execute this buildpack against OpenLiberty starter created app https://openliberty.io/start/ hoping to have src/main/liberty/config/server.xml included in the image

What did you expect to happen? I wanted to use src/main/liberty/config/server.xml as a /layers/paketo-buildpacks_open-liberty/open-liberty-runtime-full/usr/servers/defaultServer/server.xml in the image

What was the actual behavior? Please provide log output, if possible. Going through the code and docs I spotted that custom server.xml is possible with bindings. However, to me, it looks like it's only possible with bindings for the run image rather than the build image. The problem I see is that if I want to use e.g.

<featureManager> <feature>javaee-7.0</feature> <feature>microProfile-1.4</feature> </featureManager>

I need to bind server.xml when actually running the container rather than when building it. My understanding is that the recommended way of doing it (according to the OpenLiberty guide) is to include server.xml within the image. Please let me know if there is some other way of adding standard features to server.xml in this buildpack.

Build Configuration

What platform (pack, kpack, tekton buildpacks plugin, etc.) are you using? Please include a version. pack Pack: Version: 0.24.0 OS/Arch: linux/amd64

Default Lifecycle Version: 0.13.3

Supported Platform APIs: 0.3, 0.4, 0.5, 0.6, 0.7, 0.8

What buildpacks are you using? Please include versions. paketo-buildpacks/java gcr.io/paketo-buildpacks/open-liberty

paketo-buildpacks/ca-certificates 3.0.3 paketo-buildpacks/bellsoft-liberica 9.1.0 paketo-buildpacks/syft 1.7.0 paketo-buildpacks/apache-tomcat 7.1.0 paketo-buildpacks/dist-zip 5.1.0 paketo-buildpacks/spring-boot 5.5.0 paketo-buildpacks/open-liberty 0.3.2

What builder are you using? If custom, can you provide the output from pack inspect-builder <builder>? paketobuildpacks/builder:base

Can you provide a sample app or relevant configuration (buildpack.yml, nginx.conf, etc.)?

any app generated with

pack build -e BP_DEBUG --buildpack paketo-buildpacks/java --buildpack gcr.io/paketo-buildpacks/open-liberty app-name --path target/app-name

Checklist

[ ] I have included log output. n/a

[ ] The log output includes an error message. n/a

[x] I have included steps for reproduction.
Add `BP_INSTALL_TYPE` to configure install type
Summary

The proposed change is to add BP_INSTALL_TYPE option to configure which Liberty runtime we use. The value defaults to ol which just pulls Open Liberty archive as it did originally.

Three other runtime options were added:

wlp (not actually implemented yet): Download and use WebSphere Liberty runtime.

ol-stack Do not contribute any Liberty runtime in the buildpack. Instead use the OL runtime provided in the stack (requires an OL stack run image)

wlp-stack Do not contribute any Liberty runtime in the buildpack. Instead use the WLP runtime provided in the stack (requires an WLP stack run image)

Note: Two stack options were necessary since the OL and WLP use different root paths for their runtime. If we can determine which stack run image is being used during build, maybe we can combine both options to just stack.

Use Cases

Allows user to configure which runtime they'd like to use. The stack options allow users that are dependent on the existing OL / WLP images to use the Open Liberty CNB.

Checklist

[x] I have viewed, signed, and submitted the Contributor License Agreement. (IN PROGRESS)

[ ] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)

[X] I have added an integration test, if necessary.

[X] I have reviewed the styleguide for guidance on my code quality.

[X] I'm happy with the commit history on this PR (I have rebased/squashed as needed).
Fix caching for Liberty config layer.
Summary

Fixes #168. Fix caching of the base layer so that users do not need to clear the cache to update their config.

Use Cases

Allows users to update their Liberty server config without clearing their cache.

Checklist

[X] I have viewed, signed, and submitted the Contributor License Agreement.

[X] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)

[X] I have added an integration test, if necessary.

[X] I have reviewed the styleguide for guidance on my code quality.

[X] I'm happy with the commit history on this PR (I have rebased/squashed as needed).
Move linker logic into build
These changes are necessary to be compliant with stricter security settings where build and run user ID will be different.

Fixes #70, #156, #197

Summary

This change gets rid of the linker as it won't be usable with the stricter security settings coming to buildpacks. I included changes for issue 156 to include a proper default server.xml as it is necessary during the build process now (e.g. for installing server features).

Use Cases

Necessary for stricter security settings coming to buildpacks.

Checklist

[X] I have viewed, signed, and submitted the Contributor License Agreement.

[X] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)

[X] I have added an integration test, if necessary.

[X] I have reviewed the styleguide for guidance on my code quality.

[X] I'm happy with the commit history on this PR (I have rebased/squashed as needed).

The image built by paketo with kernel profile does not work

The image built by paketo with kernel profile does not work.

Expected Behavior

works

Current Behavior

not work

Steps to Reproduce

git clone https://github.com/uhryniuk/draft-guide-containerize-paketo.git
cd draft-guide-containerize-paketo/finish
mvn package

scenario 1

run following command:

pack build \
--env BP_JAVA_APP_SERVER=liberty \
--env BP_MAVEN_BUILT_ARTIFACT="target/*.war src/main/liberty/config/*" \
--env BP_LIBERTY_PROFILE=kernel \
--env BP_LIBERTY_FEATURES="cdi-3.0 jsonb-2.0 jsonp-2.0 restfulWS-3.0 jsp-2.3" \
--buildpack paketo-buildpacks/eclipse-openj9 \
--buildpack paketo-buildpacks/java \
--creation-time now \
system:1.0-SNAPSHOT

docker run --rm -d --name system -p 9080:9080 system:1.0-SNAPSHOT
visit http://localhost:9080 and have no response because the server cannot be started
docker logs system and will get the following error. Most likely the buildpacks installed incorrect set of features

{"type":"liberty_message","host":"0c3568ee7a48","ibm_userDir":"\/layers\/paketo-buildpacks_liberty\/open-liberty-runtime-kernel\/usr\/","ibm_serverName":"defaultServer","message":"CWWKF0033E: The singleton features io.openliberty.servlet.api-5.0 and io.openliberty.servlet.api-3.0 cannot be loaded at the same time.  The configured features restfulWS-3.0 and restfulWS-3.0 include one or more features that cause the conflict. Your configuration is not supported; update server.xml to remove incompatible features.","ibm_threadId":"0000002c","ibm_datetime":"2022-07-18T19:34:41.498+0000","ibm_messageId":"CWWKF0033E","module":"com.ibm.ws.kernel.feature.internal.FeatureManager","loglevel":"ERROR","ibm_sequence":"1658172881498_0000000000005"}

scenario 2

run following command by this recommendation (not using BP_MAVEN_BUILT_ARTIFACT):

pack build \
--env BP_JAVA_APP_SERVER=liberty \
--env BP_LIBERTY_PROFILE=kernel \
--env BP_LIBERTY_FEATURES="cdi-3.0 jsonb-2.0 jsonp-2.0 restfulWS-3.0 jsp-2.3" \
--buildpack paketo-buildpacks/eclipse-openj9 \
--buildpack paketo-buildpacks/java \
--creation-time now \
system:1.0-SNAPSHOT

docker run --rm -d --name system -p 9080:9080 system:1.0-SNAPSHOT
visit http://localhost:9080 and should have no problem
visit http://localhost:9080/system/properties and will get error
- Error 404: java.io.FileNotFoundException: SRVE0190E: File not found: /system/properties
run docker exec -it system cat /layers/paketo-buildpacks_liberty/open-liberty-runtime-kernel/usr/servers/defaultServer/server.xml. I found the server.xml is not from finish/src/main/liberty/config/server.xml
surely the recommendation is not the solution.

Motivations

We are writing an introduction guide but paketo buildpack does not work as expected

Symlink app archive to apps instead of expanding
Summary

Skip expanding app archive and just symlink it to reduce image size.

Use Cases

Decreases image size to reduce costs to store, transfer, etc. the image.

Checklist

[X] I have viewed, signed, and submitted the Contributor License Agreement.

[X] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)

[X] I have added an integration test, if necessary.

[X] I have reviewed the styleguide for guidance on my code quality.

[X] I'm happy with the commit history on this PR (I have rebased/squashed as needed).
Don't cache JRE
The JRE layer needs to be present at runtime but does not need to be present at build time. If required a JDK layer should be present at build time instead.

Signed-off-by: Emily Casey [email protected]

Summary

Use Cases

Checklist

[x] I have viewed, signed, and submitted the Contributor License Agreement.

[x] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)

[x] I have added an integration test, if necessary.

[x] I have reviewed the styleguide for guidance on my code quality.

[x] I'm happy with the commit history on this PR (I have rebased/squashed as needed).
Adding support to install features and fixes
Summary

Add support to install Liberty features and Liberty fixes.

Use Cases

Most liberty profiles have a subset of all available features. This PR will give the user a method to install liberty features they want to use with a smaller footprint profile. Features are specified in a space delimited list in the environment variable BP_LIBERTY_FEATURES.

iFixes are interim fixes that can be applied to a liberty installation Typically, these are security or vulnerability fixes that require immediate remediation and can not wait until the next Liberty release. This PR provides a method to install iFixes using an external configuration package.

Checklist

[x] I have viewed, signed, and submitted the Contributor License Agreement.

[ ] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)

[x] I have added an integration test, if necessary.

[x] I have reviewed the styleguide for guidance on my code quality.

[x] I'm happy with the commit history on this PR (I have rebased/squashed as needed).
Rebuild config layer if BP_LIBERTY_CONTEXT_ROOT changes
Summary

Add the value of BP_LIBERTY_CONTEXT_ROOT to the base layer metadata so that it causes config to be regenerated with the new context root value.

Use Cases

Allows users to update their context root without having to clear their cache.

Checklist

[X] I have viewed, signed, and submitted the Contributor License Agreement.

[X] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)

[X] I have added an integration test, if necessary.

[X] I have reviewed the styleguide for guidance on my code quality.

[X] I'm happy with the commit history on this PR (I have rebased/squashed as needed).
Set application location even if server.xml defines app config
Summary

The buildpack now generates app.xml if app config has been provided in the server.xml or not. This allows the Liberty buildpack to override the app location and context root (if BP_LIBERTY_CONTEXT_ROOT is provided). This change requires an ID to be set on the app config element in order for app config to be merged properly; if not set, the build will fail.

The buildpack also now checks server.xml for any app configs defined with webApplication or enterpriseApplication in addition to application.

Use Cases

Allows the buildpack to override the location and context root even if app config is provided in the server.xml.

Checklist

[X] I have viewed, signed, and submitted the Contributor License Agreement.

[X] I have linked issue(s) that this PR should close using keywords or the Github UI (See docs)

[X] I have added an integration test, if necessary.

[X] I have reviewed the styleguide for guidance on my code quality.

[X] I'm happy with the commit history on this PR (I have rebased/squashed as needed).
The file structures in the OL image bulit by paketo and ci.docker are different
The OL image built by ci.docker has following file structures

liberty runtime is located at /opt/ol/wlp

server configuration is also located at /config

logs files are located at /logs

ifixes should be put at /opt/ol/fixes/

The OL image built by Paketo

liberty runtime /layers/paketo-buildpacks_liberty/open-liberty-runtime-kernel

server configuration and logs is located at /layers/paketo-buildpacks_liberty/open-liberty-runtime-kernel/usr/servers/defaultServer

For user experience reason, it is better for the Paketo OL image to have same file structures as ci.docker does.
Implement `$BPL_OPENLIBERTY_LOG_LEVEL`
What happened?

$BPL_OPENLIBERTY_LOG_LEVEL is documented but not yet implemented. This needs to be implemented.

Checklist

[ ] I have included log output.

[ ] The log output includes an error message.

[ ] I have included steps for reproduction.

Related tags

Cloud Computing open-liberty

A Cloud Native Buildpack that contributes SDKMAN and uses it to install dependencies like the Java Virtual Machine

gcr.io/paketo-buildpacks/sdkman A Cloud Native Buildpack that contributes SDKMAN and uses it to install dependencies like the Java Virtual Machine. Be

Jan 8, 2022

A Cloud Native Buildpack that contributes the Syft CLI which can be used to generate SBoM information

gcr.io/paketo-buildpacks/syft The Paketo Syft Buildpack is a Cloud Native Buildpack that contributes the Syft CLI which can be used to generate SBoM i

Dec 14, 2022

Cloudpods is a cloud-native open source unified multi/hybrid-cloud platform developed with Golang

Cloudpods is a cloud-native open source unified multi/hybrid-cloud platform developed with Golang, i.e. Cloudpods is a cloud on clouds. Cloudpods is able to manage not only on-premise KVM/baremetals, but also resources from many cloud accounts across many cloud providers. It hides the differences of underlying cloud providers and exposes one set of APIs that allow programatically interacting with these many clouds.

Jan 11, 2022

Cloud-native way to provide elastic Jupyter Notebook services on Kubernetes

elastic-jupyter-operator: Elastic Jupyter on Kubernetes Kubernetes 原生的弹性 Jupyter 即服务介绍为用户按需提供弹性的 Jupyter Notebook 服务。elastic-jupyter-operator 提供以下特性

Dec 29, 2022

cloud-native local storage management system

Open-Local是由多个组件构成的本地磁盘管理系统，目标是解决当前 Kubernetes 本地存储能力缺失问题。通过Open-Local，使用本地存储会像集中式存储一样简单。

Dec 30, 2022

Elkeid is a Cloud-Native Host-Based Intrusion Detection solution project to provide next-generation Threat Detection and Behavior Audition with modern architecture.

Dec 30, 2022

Nocalhost is Cloud Native Dev Environment.

Most productive way to build cloud-native applications. Nocalhost The term Nocalhost originates from No Local, which is a cloud-native development too

Dec 29, 2022

cloneMAP: cloud-native Multi-Agent Platform

cloneMAP: cloud-native Multi-Agent Platform cloneMAP is a multi-agent platform (MAP) that is designed to run in a cloud environment based on Kubernete

Nov 18, 2022

TurtleDex is a decentralized cloud storage platform that radically alters the landscape of cloud storage.

TurtleDex is a decentralized cloud storage platform that radically alters the landscape of cloud storage. By leveraging smart contracts, client-side e

Feb 17, 2021

Contentrouter - Protect static content via Firebase Hosting with Cloud Run and Google Cloud Storage

contentrouter A Cloud Run service to gate static content stored in Google Cloud

Jan 2, 2022

An edge-native container management system for edge computing

SuperEdge is an open source container management system for edge computing to manage compute resources and container applications in multiple edge regions. These resources and applications, in the current approach, are managed as one single Kubernetes cluster. A native Kubernetes cluster can be easily converted to a SuperEdge cluster.

Dec 29, 2022

A Cloud Native Buildpack that provides the Open Liberty runtime

gcr.io/paketo-buildpacks/open-liberty

Behavior

Configuration

Default Configurations that Vary from Open Liberty's Default

Bindings

License

Owner

Paketo Buildpacks

Comments

Include features and ifixes in SBoM

Summary

Use Cases

Checklist

Documentation is not clear

The image built by paketo with kernel profile does not work

Unable to use custom server.xml when building the image

What happened?

Build Configuration

Checklist

Add `BP_INSTALL_TYPE` to configure install type

Summary

Use Cases

Checklist

Fix caching for Liberty config layer.

Summary

Use Cases

Checklist

Move linker logic into build

Summary

Use Cases

Checklist

The image built by paketo with kernel profile does not work

Expected Behavior

Current Behavior

Steps to Reproduce

scenario 1

scenario 2

Motivations

Symlink app archive to apps instead of expanding

Summary

Use Cases

Checklist

Don't cache JRE

Summary

Use Cases

Checklist

Adding support to install features and fixes

Summary

Use Cases

Checklist

Rebuild config layer if BP_LIBERTY_CONTEXT_ROOT changes

Summary

Use Cases

Checklist

Set application location even if server.xml defines app config

Summary

Use Cases

Checklist

The file structures in the OL image bulit by paketo and ci.docker are different

Implement `$BPL_OPENLIBERTY_LOG_LEVEL`

What happened?

Checklist

Related tags

A Cloud Native Buildpack that contributes SDKMAN and uses it to install dependencies like the Java Virtual Machine

A Cloud Native Buildpack that contributes the Syft CLI which can be used to generate SBoM information

Cloudpods is a cloud-native open source unified multi/hybrid-cloud platform developed with Golang

Cloud-native way to provide elastic Jupyter Notebook services on Kubernetes

cloud-native local storage management system

Elkeid is a Cloud-Native Host-Based Intrusion Detection solution project to provide next-generation Threat Detection and Behavior Audition with modern architecture.

Nocalhost is Cloud Native Dev Environment.

cloneMAP: cloud-native Multi-Agent Platform

TurtleDex is a decentralized cloud storage platform that radically alters the landscape of cloud storage.

Contentrouter - Protect static content via Firebase Hosting with Cloud Run and Google Cloud Storage

An edge-native container management system for edge computing

Lightweight Cloud Instance Contextualizer

Go language interface to Swift / Openstack Object Storage / Rackspace cloud files (golang)

The extensible SQL interface to your favorite cloud APIs.

Terraform provider for HashiCorp Cloud Platform.

The Cloud Posse Terraform Provider for various utilities (E.g. deep merging)

`gcr.io/paketo-buildpacks/open-liberty`