This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Development

1: How to develop
2: Website

Everything about the development of ansible-oracle.

1 - How to develop

1. How to develop in `ansible-oracle`?

Important

This is an example how most of the development in ansible-oracle is done.
A Pull-Request is only accepted, when github-Actions are valid.

1.1. Create Fork on github and contribute to the project

github has some great documentation with details about creating a fork and branch to contribute to ansible-oracle.

Important

The github-Actions will automatically start, when the branchname starts with pr or in each Pull-Request.
Please use a branch with 'pr'--prefix, when ansible-lint should be checked for each commit.

1.2. Development environment

You need an editor or IDE of your own choice and Bash as CLI. The tools for development (ansible-lint, ansible-navigator, ansible-doctor, pre-commit, …) can be provided in two ways:

Option 1: nix-shell Option 2: local Ansible-Controller

	Option 1: nix-shell	Option 2: local Ansible-Controller
Setup	Install Nix once, then `nix-shell` in the repository	Create and maintain venvs and tools manually
Tools	All tools and venvs are provided automatically	Each tool is installed by yourself
Recommended for	All developers	Developers with an existing Ansible-Controller or experience with Docker

Setup

Install Nix once, then nix-shell in the repository

Create and maintain venvs and tools manually

Tools

All tools and venvs are provided automatically

Each tool is installed by yourself

Recommended for

All developers

Developers with an existing Ansible-Controller or experience with Docker

Tip	`nix-shell` is the recommended way for developing in `ansible-oracle`. Choose one option, the chapter Working with the tools is the same for both.

1.2.1. Option 1: nix-shell (recommended)

One command in the repository provides all tools, which are needed for development, tests and releases.

1.2.1.1. What is nix-shell?

Nix is a package manager for Linux and macOS, which works independent of the package manager of the OS (dnf, apt, …). All packages are installed in /nix/store, each version in its own directory. Different versions of a tool can be installed in parallel without conflicts, the OS installation is not changed.

nix-shell starts a new shell with the packages from shell.nix in the repository. The tools are only available inside this shell, exit returns to the normal shell without any change in the OS. Packages are downloaded once and reused in all later sessions.

The shell.nix of ansible-oracle provides:

Tools from Nix: python313, shellcheck, go-task (task), jq, yq, gh (GitHub CLI), renovate, tig, ncdu and wslu for WSL2.

Note	`gh` is only needed for creating releases (`task release:*`) and for running renovate against a fork. This is only relevant for a few maintainers. For normal development and Pull-Requests `gh` is not needed and no login with `gh auth login` is required.

Python venvs in .venv/ of the repository, which are created automatically from tools/dev/requirements_*.txt at the first start:

venv Usage

venv	Usage
`.venv/ansible-oracle`	`ansible-lint`, `ansible-navigator`, `molecule`, `pre-commit` - activated in the shell
`.venv/ansible-doctor`	`ansible-doctor` (has conflicts with `ansible-lint` in the same venv)
`.venv/antsibull`	`antsibull-changelog` for changelog fragments and releases
`.venv/ansible-2.16`	tests with `ansible-core` 2.16
`.venv/ansible-2.19`	tests with `ansible-core` 2.19

.venv/ansible-oracle

ansible-lint, ansible-navigator, molecule, pre-commit - activated in the shell

.venv/ansible-doctor

ansible-doctor (has conflicts with ansible-lint in the same venv)

.venv/antsibull

antsibull-changelog for changelog fragments and releases

.venv/ansible-2.16

tests with ansible-core 2.16

.venv/ansible-2.19

tests with ansible-core 2.19

1.2.1.2. Why nix-shell instead of containers?

Containers (see docker/) are an alternative, but they need much more effort in daily work:

No image build, no container runtime: There is no Dockerfile to maintain and no daemon to start. nix-shell works on every Linux, WSL2 and macOS with Nix installed.
Work directly in your repository: Editor, IDE and git work on the files as usual. There are no volume mounts and no problems with file owners or UIDs between container and host.
Your credentials are available: SSH agent and keys and the git configuration of your user are used directly, for maintainers also the login of gh. A container needs extra mounts for all of them.
pre-commit works as on GitHub: The hooks of pre-commit use the Python and tools from nix-shell. E.g. the ansible-lint hook fails outside of nix-shell with the Python of an older OS.
Same tools for all developers: Everybody uses the same tools and pip requirements from the repository. No manual installation of shellcheck, gh, jq, task or the venvs is needed.
All tasks work out of the box: task for molecule, renovate and releases (task release:pr, …) needs exactly the tools from nix-shell.
Nothing is left in your OS: The tools are removed from PATH when the shell is left. The venvs are inside the repository in .venv/ and are not part of git.

1.2.1.3. Install Nix

Nix is installed once per machine, see nixos.org/download. The multi-user installation is recommended (needs sudo only for the installation):

sh <(curl --proto '=https' --tlsv1.2 -L https://nixos.org/nix/install) --daemon

Note	For WSL2 the multi-user installation needs `systemd` in WSL2 (`/etc/wsl.conf`: `[boot]` `systemd=true`).

1.2.1.4. Using nix-shell

cd ansible-oracle
nix-shell

The first start takes some minutes, because packages are downloaded and the venvs are created. Afterwards nix-shell starts in seconds and .venv/ansible-oracle is activated.

# check the tools
task --list
ansible-lint --version

# leave nix-shell
exit

A single command can be executed without an interactive shell:

nix-shell --run 'task release:pr'

Tip	With direnv and nix-direnv the environment is loaded automatically, when you change into the repository. The repository contains a `.envrc` for this, which must be allowed once with `direnv allow`.

1.2.1.5. Update the venvs

The venvs are only created, when the directory in .venv/ does not exist. After a change of tools/dev/requirements_*.txt (e.g. by renovate) the venv must be updated manually. Errors like ModuleNotFoundError: No module named 'yaml' show an incomplete venv.

# update a venv in place
.venv/antsibull/bin/pip --require-virtualenv install -r tools/dev/requirements_antsibull.txt

# or recreate it with the next start of nix-shell
rm -rf .venv/antsibull
nix-shell

Note	`shell.nix` uses the `nixpkgs` channel of your system, the versions of the Nix tools can differ between developers. The versions of the Python tools are pinned in `tools/dev/requirements_*.txt`.

1.2.2. Option 2: local Ansible-Controller

Note	This option is only for developers, who want to use an existing Ansible-Controller or a container and have experience with venvs and Docker. All tools must be installed and updated manually. `nix-shell` does all of this automatically, see Option 1: nix-shell (recommended).

The examples use venvs in ~/venv. Make sure to use a current Python 3, the pinned versions in tools/dev/requirements_*.txt need a recent Python. The activate script is used to change the Python environment in current shell.

1.2.2.1. venv for ansible-oracle

The venv contains ansible-lint, ansible-navigator, molecule and pre-commit.

configure venv for ansible-oracle

python3 -m venv ~/venv/ansible-oracle
. ~/venv/ansible-oracle/bin/activate
pip --require-virtualenv -v install -r tools/dev/requirements_dev.txt

Start ansible-navigator

. ~/venv/ansible-oracle/bin/activate
ansible-navigator --version

1.2.2.2. venv for ansible-doctor

Important

ansible-doctor has module dependencies with conflicts between ansible-lint. We have to use a dedicated venv for ansible-doctor.

configure venv for ansible-doctor

python3 -m venv ~/venv/ansible-doctor
. ~/venv/ansible-doctor/bin/activate
pip --require-virtualenv -v install -r tools/dev/requirements_doctor.txt

Start ansible-doctor

~/venv/ansible-doctor/bin/ansible-doctor --version

1.2.2.3. Ansible-Container

Instead of local venvs, the container from docker/ can be used as Ansible-Controller.

cd ansible-oracle/docker
docker-compose run --rm -w /git/ansible-oracle/example/beginner/ansible ansible bash

Note	Some `pre-commit` hooks (e.g. `ansible-lint`) need a recent Python and can fail with the Python of an older OS.

1.3. Working with the tools

The following chapters are the same for both options. With nix-shell the tools are in PATH, with a local Ansible-Controller the venv must be activated before.

1.3.1. pre-commit

Important

pre-commit is a nice tool for git.
Each commit is locally checked against multiple rules, to prevent failures in github-Actions after pushing commits to github.
This tool is recommended but not mandatory for contributing to ansible-oracle.

Install the git hook once (cd into repository before)

pre-commit install

Important

Do not forget to place change files in stage area of git before starting pre-commit.
pre-commit works only on the data of the stage area!

run outside git

pre-commit run

1.3.2. ansible-doctor

Start ansible-doctor with pre-commit

pre-commit run ansible-doctor

Start ansible-doctor to create new README.md file

cd ansible-oracle

# nix-shell
.venv/ansible-doctor/bin/ansible-doctor -f -r roles

# local Ansible-Controller
~/venv/ansible-doctor/bin/ansible-doctor -f -r roles

1.3.3. Start Playbook

Important

Don’t forget to set the working branch in requirements.yml.
Remove/Disable devsec.hardening for faster installation of Ansible collection, when devsec should not be tested.

Edit requirements.yml

---
collections:
  # - name: devsec.hardening
  #   version: 8.2.0
  - name: https://github.com/Rendanic/ansible-oracle.git
    type: git
    version: devbranch

Execute ansible-playbook

ansible-galaxy collection install --force -r requirements.yml && ansible-playbook -i inventory/hasfd -e hostgroup='all' playbooks/single-instance-asm.yml --limit '*19c*'

2. Pull-Requests

2.1. Important Information

The ansible-oracle project introduced antsibull-changelog for managing the CHANGELOG.rst based on fragments in changelogs/fragments.

The ID should point to the PR and the filename describe the PR in short form. The fragments are part of the PR. If multiple PRs are open, the upper rule makes sure that no duplicate files are created during merge.

Important

Each Pull-Requests needs a fragment from Release 3.0.0 onwards!

2.2. Create Pull-Requests with Claude Code

tools/claude/skills/pr contains the skill /pr for Claude Code. It creates a Pull-Request from the current state of the local working copy: create a branch, commit, add a changelog fragment, push into the fork and run gh pr create.

2.2.1. Setup

gh is part of shell.nix and must be logged in once (see Setup gh for renovate). Without a gh in the PATH the skill uses nix-shell -p gh.

Install the skill as personal skill for Claude Code:

mkdir -p ~/.claude/skills
ln -sfn "$(pwd)/tools/claude/skills/pr" ~/.claude/skills/pr

Note	The symlink points into the working copy. `/pr` is missing on branches without `tools/claude`. Use `cp -r` instead of `ln -sfn` for an independent copy.

2.2.2. Usage

/pr [draft] [base=<branch>] [milestone=<pattern>] [label=<p1,p2,...>] [notes on content]

Argument Description

Argument	Description
`draft`	Create the Pull-Request as draft.
`base=<branch>`	Target branch. Default: default branch of the target repository.
`milestone=<pattern>`	Milestone in the target repository. The value is searched as case-insensitive regular expression in the titles of the open milestones (e.g. `milestone=5.0` or `milestone=5.0.*alpha`). A number is the number of the milestone.
`label=<p1,p2,…>`	Comma-separated labels in the target repository. Every entry is searched as case-insensitive regular expression in the label names (e.g. `label=enh,bug,break`).
any text	Hints for title and description of the Pull-Request.

draft

Create the Pull-Request as draft.

base=<branch>

Target branch. Default: default branch of the target repository.

milestone=<pattern>

Milestone in the target repository. The value is searched as case-insensitive regular expression in the titles of the open milestones (e.g. milestone=5.0 or milestone=5.0.*alpha). A number is the number of the milestone.

label=<p1,p2,…>

Comma-separated labels in the target repository. Every entry is searched as case-insensitive regular expression in the label names (e.g. label=enh,bug,break).

any text

Hints for title and description of the Pull-Request.

Examples

/pr milestone=4.17 label=bug
/pr draft milestone=5.0.*alpha label=enh,break

2.2.3. Behavior

The target is the remote upstream, if it exists (Pull-Request from <fork-owner>:<branch> into oravirt/ansible-oracle), otherwise origin.
On the default branch a new branch is created.
A missing fragment in changelogs/fragments is created. The PR reference is added later with tools/release/fix_fragments.py.
Labels and milestones are resolved against the target repository before pushing. Only open milestones are considered. An exact name wins, otherwise the value is searched as regular expression. The skill stops when a value matches nothing or more than one name (the matches are listed), nothing is created. Setting labels or milestones in oravirt/ansible-oracle needs the triage permission.
An existing Pull-Request for the branch is updated (push, labels, milestone) instead of creating a new one.
The skill asks before a force push, a push into the default branch, unrelated uncommitted changes or merge conflicts.

Important

Invoking /pr allows Claude to push the branch and to create the Pull-Request without further confirmation.

3. Create new Releases in `ansible-oracle`

Important

This is only needed for creating new Releases in ansible-oracle - not for development!

3.1. Install antsibull-changelog

cd ansible-oracle/docker
docker-compose run --rm -w /git/ansible-oracle/example/beginner/ansible ansible bash

pip3 install antsibull-changelog

3.2. Working with antsibull-changelog

Changelogs for Collections: https://github.com/ansible-community/antsibull-changelog/blob/main/docs/changelogs.rst#releasing-a-new-version-of-a-collection

3.3. Add PR references to fragments

Each entry in a fragment should end with a reference to its Pull-Request, e.g. (oravirt#599). tools/release/fix_fragments.py adds missing references to all fragments in changelogs/fragments.

The Pull-Request is determined from the commits of the fragment file via the GitHub API, newest commit first. A merged Pull-Request of any of these commits wins over an open one. The open Release-Pull-Request is only used for fragments, which are not part of a merged Pull-Request. The reference is appended to every entry that does not end with it, even if the entry already references an issue.

# show what would be changed
tools/release/fix_fragments.py --dry-run

# update the fragments
tools/release/fix_fragments.py

Single fragments can be checked with -f / --fragment, e.g. by a developer after the Pull-Request for a new fragment has been created. The parameter accepts a path or a filename from changelogs/fragments and can be repeated. With --dry-run the script returns exit code 1 if the fragment needs to be updated.

# check a single fragment
tools/release/fix_fragments.py --dry-run -f blanks-in-gi-responsefile.yml

# update a single fragment
tools/release/fix_fragments.py -f changelogs/fragments/blanks-in-gi-responsefile.yml

Note	The Pull-Request must exist on GitHub, otherwise the fragment is reported as `no pull request found`.

Fragments without a Pull-Request and entries that cannot be updated automatically (e.g. broken quoting) are reported and must be fixed manually. The script returns exit code 1 in this case.

Without --fragment all fragments in changelogs/fragments are processed.

Note

Without a token the GitHub API allows only 60 requests per hour. The token is taken from GITHUB_TOKEN or GH_TOKEN. Without these variables the token stored by gh is used, when all fragments are processed (gh auth token or ~/.config/gh/hosts.yml of the active account). An invalid token is ignored with a warning.

This must be done before antsibull-changelog release, because the release removes the fragments.

3.4. Collection version in requirements.yml

The examples in example/*/ansible/requirements.yml must reference the collection version from galaxy.yml. The same is needed for the template tools/release/requirements.yml. tools/release/galaxy_version.sh checks or sets the version in all these files.

# check the versions
tools/release/galaxy_version.sh check

# set the version from galaxy.yml
tools/release/galaxy_version.sh fix

check returns exit code 1 if a file does not use the version from galaxy.yml. fix changes only the version of the collection entry, comments in the files are kept.

The script is also available as task example:check and task example:fix from the repository root.

3.5. Creating new releases

tools/release/build_release.sh executes the whole release process:

-m pr: Create the Release-Pull-Request with changelog.
-m release: Create the tag and a draft release for the Release-Pull-Request.
-m galaxy: Sync the branch of a published release in your fork and deploy the collection to Ansible Galaxy.
-m docs: Update the website, without creating a release.

Note	The whole release process should be done with a dedicated Pull-Request.

3.5.1. Release branches

Releases are built from master or from a maintenance branch release/<major>.x (e.g. release/4.x) for an older major version. The branch is the base of the Release-Pull-Request:

The release branch pr/<version> must be created from the base branch, e.g. git switch -c pr/4.17.1 upstream/release/4.x.
Without -b the base branch is taken from the version: master, when the major version is the same or higher than in galaxy.yml of master, otherwise release/<major>.x. Use -b, when a release for the same major version as master is built from a maintenance branch (e.g. -b release/4.x, while master is still 4.x).
The script aborts, when pr/<version> is not based on the current base branch. This prevents a Pull-Request with commits of another branch.
Only releases from master are marked as latest release on GitHub.
The website is only updated for releases from master.
antsibull-changelog writes the changelog of the base branch only. A release from a maintenance branch is not part of the changelog in master, until changelogs/changelog.yaml is merged.

Requirements:

The release branch is named pr/<major>.<minor>.<patch>, e.g. pr/4.16.0, and is created from the base branch.
origin is your fork on GitHub.
gh is logged in (gh auth status), jq and curl are installed and .venv/antsibull exists. gh, jq and the venv are provided by nix-shell.
The script is executed in nix-shell, because the pre-commit hooks of the version commit need it.

nix-shell

# release from master
git switch -c pr/4.18.0 upstream/master
tools/release/build_release.sh -m pr

# release from a maintenance branch
git switch -c pr/4.17.1 upstream/release/4.x
tools/release/build_release.sh -m pr -b release/4.x

Use -n (dry-run) to check all steps before: nothing is changed locally or on GitHub, the commands are shown instead.

3.5.2. Release-Pull-Request (`-m pr`)

The script executes the following steps:

Determine the base branch (-b or from the version, see Release branches) and check that the release branch is based on it.
Read the version from the branch name (pr/4.16.0 → 4.16.0) and set it in galaxy.yml.
Set the version in all requirements.yml with tools/release/galaxy_version.sh fix.
Commit galaxy.yml and all requirements.yml as Release <version>. This is the initial commit for the Pull-Request, GitHub refuses a Pull-Request without commits. Other changes in the working tree and the index are not part of this commit. The step is skipped, when the version is already committed.
Push the branch to origin.
Create the Pull-Request Release <version> against the base branch in oravirt/ansible-oracle. An existing open Pull-Request of the branch is used, the script aborts when it has another base branch.
Add the PR references to the fragments with tools/release/fix_fragments.py. The script is aborted, when fragments cannot be updated.
Create the changelog with .venv/antsibull/bin/antsibull-changelog release. antsibull-changelog reads the version from galaxy.yml and aborts, when the release already exists in CHANGELOG.rst. The fragments are removed.
Replace the description of the Pull-Request with the release notes of the new version (Markdown, antsibull-changelog generate --only-latest).

The changes of fragments and changelog are not committed by the script. Review them, then commit and push them to the release branch.

3.5.3. Draft release (`-m release`)

tools/release/build_release.sh -m release

The script executes the following steps:

Search the Pull-Request of the release branch from your fork, open or merged.
Create the tag v<version> (e.g. v4.16.0) like the existing release tags. The tag points to the merge commit of the Pull-Request. The Pull-Request must be merged before, otherwise the script is aborted: a rebase merge creates new commits, so a tag on the last commit of the open Pull-Request would not be part of the branch. Only for tests against a fork (-r) the last commit of an open Pull-Request is used.
Push the tag to the git remote of the GitHub repository (e.g. upstream for oravirt/ansible-oracle). An existing tag is used, when it points to the same commit, otherwise the script is aborted.
Create the draft release Release <version> with the description of the Pull-Request as text. The text of an existing draft release is updated, an already published release aborts the script. Releases from master are created with --latest, releases from a maintenance branch with --latest=false.
Update the website with tools/release/build_website.sh -m push, see website.adoc. The website is built from the local checkout of the release branch and published at once. This step is skipped for releases from a maintenance branch and for tests against a fork (-r other than oravirt/ansible-oracle).

The draft release must be published manually on GitHub. For a release from a maintenance branch, check that "Set as the latest release" is not selected when publishing.

Note	The website step needs `docker`, `rsync` and `task` and committed sources in `website/`, `doc/`, `roles/` and `docker/hugo/`. When it fails, fix the problem and run `-m release` again: the existing tag and draft release are reused.

3.5.4. Update the website (`-m docs`)

tools/release/build_release.sh -m docs

Builds and publishes the website with tools/release/build_website.sh -m push, like the last step of -m release. Use it to update the website without a release, e.g. after changes in doc/ or website/, or to repeat a failed website step of -m release. No release branch is needed, the website is built from the current local checkout. The step is skipped with -r other than oravirt/ansible-oracle.

For more options (commit only, Pull-Request in the website repository) use tools/release/build_website.sh directly, see website.adoc.

3.5.5. Sync fork and deploy to Ansible Galaxy (`-m galaxy`)

# latest published release
tools/release/build_release.sh -m galaxy

# release from a maintenance branch
tools/release/build_release.sh -m galaxy -t v4.17.1

GitHub syncs the fork in the background. -m galaxy forces the sync at once, to deploy a release merged in upstream without waiting. The fork is <gh user>/ansible-oracle, the user is taken from the login of gh.

Take the release from -t or the latest published release of the upstream repository. Draft releases are ignored, so the release must be published before. A release from a maintenance branch is not the latest release, use -t for it.
Search the branch with the content of the release as last commit: master or release/<major>.x. The content (git tree) is compared and not the commit, because a rebase merge creates new commits with the same content. When the branch has newer changes than the release, nothing is deployed.
Sync this branch in the fork with the upstream repository (gh repo sync). When the branch does not exist in the fork (e.g. release/4.x), it is created at the release commit.
Start the workflow Deploy Collection in the fork on this branch, which uploads the collection to Ansible Galaxy. The script waits for the workflow and shows the progress of the steps (gh run watch). When the workflow fails, the log of the failed steps is shown and the script returns exit code 1. The workflow is not started, when the version is already on Ansible Galaxy, so -m galaxy can be repeated safely.

The mode works only on GitHub with gh (and the API of Ansible Galaxy), nothing is changed locally and no release branch is needed. The sync is refused, when the branch of the fork contains own commits - no hard reset is done.

3.5.6. Tasks

The modes are also available as tasks from the repository root. Options are passed after --.

task release:pr        # -m pr
task release:draft     # -m release (incl. website)
task release:docs      # -m docs
task release:galaxy    # -m galaxy

The creation of the Release-Pull-Request can be tested against your fork:

task release:pr -- -i -r <github user>/ansible-oracle

The Pull-Request is then created in your fork instead of oravirt/ansible-oracle.

Warning

fix_fragments.py searches the Pull-Requests of the fragments in your fork and not in upstream. The fragments are part of Pull-Requests in oravirt/ansible-oracle, so the results are wrong: fragments get the reference of the test Pull-Request in your fork (e.g. Rendanic#93) or are skipped with no pull request found. -i is needed, otherwise the script stops at the skipped fragments. Changelog and description of the test Pull-Request contain these wrong references, never use them for a real release. Revert all changes in the working tree after the test.

3.5.7. Options

-m pr|release|galaxy|docs: Mode of the script, see above.
-b <branch>: Base branch of the Release-Pull-Request (only -m pr, default: from the version, see Release branches).
-t <tag>: Release for -m galaxy, e.g. v4.17.1 (default: latest published release).
-n: Dry-run: show the commands, which would change something locally or on GitHub, without executing them. fix_fragments.py runs with --dry-run.
-r <owner/repo>: GitHub repository for the Pull-Request, the release, the lookup of the fragment Pull-Requests and source of the fork sync (default: oravirt/ansible-oracle).
-i: Continue, when fragments cannot be updated by fix_fragments.py (only -m pr).

A test against your fork is possible with -i -r <your account>/ansible-oracle, or -m release -r <your account>/ansible-oracle. The fragments get the reference of the Release-Pull-Request in the fork or are skipped, because the Pull-Requests of the fragments are in oravirt/ansible-oracle. Revert all changes in the working tree after such a test and remove the test tag and draft release (git tag -d v<version>, the tag and the release in your fork).

4. Check dependencies with renovate

Important

This is optional and not needed for the normal development in ansible-oracle.
It is used by maintainers to find updates for pip modules and container images.

renovate is part of shell.nix. The configuration is in renovate.json, the tasks are in tools/renovate/Taskfile.yml.

4.1. Local checks

The local checks read the working tree and do not change anything. No GitHub setup is needed. renovate.json is read from the working tree, but it must be tracked by git.

task renovate:check     # pip modules, container images, collections and github actions
task renovate:pip       # pip modules and pre-commit hooks
task renovate:docker    # container images only
task renovate:galaxy    # collections in galaxy.yml and requirements.yml
task renovate:report    # show the result of the last run again
task renovate:validate  # validate renovate.json
task renovate:ocenv     # check roles/oradb_manage_db/files/ocenv for local changes

The report shows only dependencies with updates. Add ONLY_UPDATES=false to show all dependencies including up-to-date and disabled, e.g. task renovate:check ONLY_UPDATES=false or task renovate:report ONLY_UPDATES=false.

Dependencies which are pinned intentionally (e.g. ansible-core or the molecule base images) are disabled with packageRules in renovate.json. They are shown as disabled in the report.

pip modules are checked against the python version, which installs the requirements file (constraints and constraintsFiltering in renovate.json). Versions, which do not support this python version (requires_python on PyPI), are not proposed:

python 3.13: venvs from shell.nix (requirements_dev_2.*.txt, requirements_doctor.txt, requirements_antsibull.txt)
python 3.11: execution environment (execution-environment.yml) and shell.nix use requirements_dev.txt, the lowest version is used.

Change the python version in renovate.json, when the python version in shell.nix or execution-environment.yml is changed. Dependencies between pip modules (e.g. a version of ansible-lint which needs a newer ansible-core) are not checked by renovate.

The collections in dependencies of galaxy.yml are checked against Ansible Galaxy. community.general and community.crypto are disabled, because the version ranges are set intentionally.

Renovate does not know requires_ansible of a collection and proposes versions, which need a newer ansible-core than requires_ansible in meta/runtime.yml. task renovate:galaxy therefore runs tools/renovate/galaxy_requires.py (task renovate:galaxy:requires) after the renovate check. It shows requires_ansible of each dependency in galaxy.yml and the newest version which can be installed with the minimum ansible-core version. The script fails, when a dependency needs a newer ansible-core. Only update to the version in column COMPATIBLE.

Renovate does not check the dependencies of a collection either. galaxy_requires.py checks the dependencies of every version against the version ranges in galaxy.yml and the minimum ansible-core version. COMPATIBLE is the newest version without a conflict, the script fails and shows the conflicting dependency, when the version in galaxy.yml has a conflict. Only one level is checked (dependencies of dependencies are not checked), test the installation with task molecule:collection after an update. opitzconsulting.ansible_oracle in requirements.yml is disabled, because the version is set by tools/release/galaxy_version.sh (see Collection version in requirements.yml).

Tools which are used in requirements, in .pre-commit-config.yaml and in the github workflows (ansible-doctor, ansible-lint, antsibull-changelog) are grouped with groupName. An update changes all files in one branch, so the versions are always the same. The github workflow for ansible-doctor installs tools/dev/requirements_doctor.txt. The versions of github actions (e.g. ansible/ansible-lint) and of pip install <module>==<version> in the workflows are checked as well (only with task renovate:check).

The pre-commit hooks are looked up with GitHub tags, which needs a token. The local checks use the token from gh auth token, if gh is logged in.

4.1.1. ocenv from oracle-scripts

roles/oradb_manage_db/files/ocenv is a copy of ocenv from opitzconsulting/oracle-scripts. The file is part of the collection, so no online access is needed during the installation.

Important

Do not change roles/oradb_manage_db/files/ocenv in ansible-oracle. Changes are made in oracle-scripts, the file is replaced only by renovate.

The upstream repository has no releases. The version is the committer date of the last commit of ocenv in UTC (YYYY.MMDD.HHMMSS, custom datasource ocenv in renovate.json).
tools/renovate/ocenv.version holds the version, the upstream commit and the git blob sha of the file.
For an update renovate runs tools/renovate/ocenv_update.sh as postUpgradeTasks. The script downloads ocenv of the matching upstream commit, compares git hash-object with the blob sha from GitHub and replaces the file. ocenv, tools/renovate/ocenv.version and the changelog fragment are committed by renovate into the update branch.
The git blob sha is the same in oracle-scripts and ansible-oracle. task renovate:ocenv compares the local file with the blob in tools/renovate/ocenv.version and fails after a local change, which would be overwritten by the next update.

4.1.2. Oracle Release Updates for molecule

The molecule scenarios use the current Release Update (RU) and the previous one (RU-1) for the patch tests. Both are set in extensions/molecule/shared_config/inventory/group_vars/all/oracle_ru.yml and updated by renovate:

Variable Example Custom datasource in renovate.json

Variable	Example	Custom datasource in `renovate.json`
`molecule_ru_19`	`19.33`	`oracle-ru-19`
`molecule_ru_19_prev`	`19.32`	`oracle-ru-19-prev`
`molecule_ru_26`	`23.26.4`	`oracle-ru-26`
`molecule_ru_26_prev`	`23.26.3`	`oracle-ru-26-prev`

molecule_ru_19

19.33

oracle-ru-19

molecule_ru_19_prev

19.32

oracle-ru-19-prev

molecule_ru_26

23.26.4

oracle-ru-26

molecule_ru_26_prev

23.26.3

oracle-ru-26-prev

The versions are derived from the quarterly Critical Patch Updates in the Oracle Security Alerts feed. *-prev is the same calculation, shifted by one quarter.
The RU of the database is used for Grid Infrastructure / Oracle Restart as well.
Short forms for ORACLE_HOME, gold images and oracle_home_name are derived in oracle_ru.yml: molecule_ru_19_nodot (1933), molecule_ru_26_short (26.04), molecule_ru_26_nodot (2604), molecule_ru_26_gi (26_04, gold image of Grid Infrastructure) and the same for *_prev. Use the variables in the scenarios instead of fixed versions.
The keys of db_homes_config are fixed, only the values (oracle_home, imagename, oracle_home_name) follow the RU: db19curr-si-ee, db19prev-si-ee, db26curr-si-ee, db26prev-si-ee in group_vars/all/oracle_db.yml. db19-si-ee (RU-1, patch tests) and db26-si-ee (current RU) are aliases for the scenarios without a specific RU.
Out-of-Place Patching tests (dbfs-sh, has-ol9, rac) install RU-1 in the converge and switch to the current RU in side_effect.yml.
All 4 variables are updated in one branch pr/renovate/oracle-ru-molecule.
Manual steps after an update (listed in the Pull-Request as well): create the gold images for the new RUs and update the RU patches in oracle_sw_patches. renovate does not know patch ids and file names of the RUs.
Keep the comment # renovate: datasource=… depName=… directly above the variable.

4.2. Setup gh for renovate

Important

The GitHub CLI gh is only needed for running renovate against your fork on GitHub (task renovate:fork) and for the Claude Code skill /pr (see Create Pull-Requests with Claude Code).
The local checks work without it.

gh is part of shell.nix. Login once with your GitHub account:

gh auth login     # GitHub.com, HTTPS, login with a web browser
gh auth status

The token of gh needs the scope repo for pushing branches into your fork and workflow for updates in .github/workflows. Both are part of the default scopes of gh auth login.

task renovate:fork takes the token from gh auth token for each run. It is passed only to renovate and not stored anywhere. A token in RENOVATE_TOKEN is used instead, if it is set.

4.3. Run renovate against your fork

renovate clones your fork from GitHub, creates a branch for each update and pushes it together with a Pull-Request into your fork. The configuration is taken from renovate.json in FORK_BASE of the fork, not from your working tree. renovate.json, tools/renovate/changelog_fragment.sh and tools/renovate/ocenv_update.sh must be pushed into FORK_BASE before the first run. Without renovate.json renovate skips the fork (disabled-no-config).

FORK_REPO is the fork on GitHub, FORK_BASE the branch for the updates (default: master).

# show what would be done
task renovate:fork:dry-run FORK_REPO=<github user>/ansible-oracle

# create branches and Pull-Requests in the fork
task renovate:fork FORK_REPO=<github user>/ansible-oracle

# with milestone and labels for the Pull-Requests
task renovate:fork FORK_REPO=<github user>/ansible-oracle M=<milestone> L=<label>,<label>

Optional parameters for the Pull-Requests:

M: number or title of a milestone in FORK_REPO. The task fails, when the title does not exist.
L: comma separated list of labels. Missing labels are created by GitHub.

Milestone and labels are set, when renovate creates a Pull-Request.

The tasks have no hourly limit for new Pull-Requests (prHourlyLimit is 2 in config:recommended), each run creates all Pull-Requests. The limit of 10 open Pull-Requests (prConcurrentLimit) is still active.

The branches can be tested locally after git fetch origin.

The branches are named pr/renovate/<update>, so the github-Actions are started for each push. Each update adds a changelog fragment changelogs/fragments/renovate-<dependency>.yml in section minor_changes, created by tools/renovate/changelog_fragment.sh. Grouped updates (e.g. ansible-lint in requirements, pre-commit and workflow) add one fragment renovate-<group>.yml for the whole group. The PR reference is added later with tools/release/fix_fragments.py.

Note	renovate keeps the clones and its cache in `~/.cache/renovate`. The clone is reset by each run, do not use it as working copy.

4.4. Run renovate against upstream

Important

Only for maintainers with write access to oravirt/ansible-oracle.
The branches and Pull-Requests are created in the upstream repository and are public.

renovate runs directly on the upstream repository (UPSTREAM_REPO, default oravirt/ansible-oracle) and UPSTREAM_BASE (default master). The branches pr/renovate/<update> are pushed into the upstream repository, so they do not collide with the branches of task renovate:fork in your fork. The configuration is taken from renovate.json in UPSTREAM_BASE.

# show what would be done
task renovate:upstream:dry-run

# create branches and Pull-Requests in upstream
task renovate:upstream M=<milestone> L=<label>,<label>

The task fails, when your GitHub account has no write access to UPSTREAM_REPO. M and L work as described for task renovate:fork, the milestone is searched in UPSTREAM_REPO.

5. Update Vagrantbox on vagrantcloud for examples

Important

This is only an example how to do it, when custom images should be used.

5.1. Create new Box

git clone https://github.com/Rendanic/bento.git
cd bento
git checkout oc
cd packer_templates/oraclelinux/
./oc_oracle-7.x_vbox.sh oracle-7.9-x86_64.json

5.2. Upload new Version

Important

Change the boxname to an account with write privileges.

Upload new version after build:

boxname="Rendanic/oraclelinux-7.x"
vagrantversion=$(date +%y%m%d)

vagrant cloud version create ${boxname} "$vagrantversion"
vagrant cloud provider create ${boxname} virtualbox "$vagrantversion"
vagrant cloud provider upload ${boxname} virtualbox "$vagrantversion" ../../builds/oracle-7.9.virtualbox.box
vagrant cloud publish --release ${boxname} "$vagrantversion" virtualbox

6. ansible molecule

Warning

The timeout for pulling docker Container in molecule is 15s at the moment.
There is no parameter to increase the timeout. Pull container manually before starting the scenario. The image url is defined in molecule.yml.

The following scenarios are existing in the `ansible-oracle profect.

Stage Description

Stage	Description
dbfs	Full stage for installation of RDBMS 19c + Patching and create database, AHF registered as client in AHF Fleet Insights of the scenario `ahf` (see Oracle Autonomous Health Framework (AHF)). Requires installation media in /vagrant on local host.
dbfs-ol9	Full stage for installation of RDBMS 19c + Patching on OL9 and create database. Requires installation media in /vagrant on local host.
dbfs-ol10	Same as `dbfs` on OL10: Oracle AI Database 26ai with module tests. Requires installation media in /vagrant on local host.
dataguard	Data Guard on OL10 with Oracle AI Database 26ai: 2 containers `dataguard-1` (primary) and `dataguard-2` (standby created by `oradb_standby`), broker configuration `DG1`. Side-effect: switchover tests, see Data Guard tests. Requires installation media in /vagrant on local host.
default	Stage for testing common and orahost roles with different Distributions.
download	Download Patches from support.oracle.com for dbfs stage.
golden	Create Golden-Image for installation from dbfs.
ahf	AHF Fleet Insights with repository in a container with Oracle Database Free and AHF registered as client: 2 containers `ahf` and `ahf-db`, network `molecule-ahf`. Requires a MOS login (downloads) and `AHFFI-LINUX_v*.zip`, see Oracle Autonomous Health Framework (AHF).

dbfs

Full stage for installation of RDBMS 19c + Patching and create database, AHF registered as client in AHF Fleet Insights of the scenario ahf (see Oracle Autonomous Health Framework (AHF)). Requires installation media in /vagrant on local host.

dbfs-ol9

Full stage for installation of RDBMS 19c + Patching on OL9 and create database. Requires installation media in /vagrant on local host.

dbfs-ol10

Same as dbfs on OL10: Oracle AI Database 26ai with module tests. Requires installation media in /vagrant on local host.

dataguard

Data Guard on OL10 with Oracle AI Database 26ai: 2 containers dataguard-1 (primary) and dataguard-2 (standby created by oradb_standby), broker configuration DG1. Side-effect: switchover tests, see Data Guard tests. Requires installation media in /vagrant on local host.

default

Stage for testing common and orahost roles with different Distributions.

download

Download Patches from support.oracle.com for dbfs stage.

golden

Create Golden-Image for installation from dbfs.

ahf

AHF Fleet Insights with repository in a container with Oracle Database Free and AHF registered as client: 2 containers ahf and ahf-db, network molecule-ahf. Requires a MOS login (downloads) and AHFFI-LINUX_v*.zip, see Oracle Autonomous Health Framework (AHF).

6.1. Directory for installation media and patches

The scenarios dbfs, dbfs-ol9, dbfs-ol10, dataguard, download, golden and ahf mount a local directory as /vagrant into the container. The local directory must be defined by MOLECULE_SW_DIR, there is no default. The path inside the container is always /vagrant.

Set the directory as environment variable

export MOLECULE_SW_DIR=/mnt/d/ORACLE/sw/linux64

or permanently in extensions/.env (ignored by git)

MOLECULE_SW_DIR=/mnt/d/ORACLE/sw/linux64

extensions/.env contains KEY=VALUE lines in shell syntax (no YAML). It is loaded by the Taskfile and tools/regression.sh with tools/load_env.sh. Variables already set in the environment have precedence over the file.

Important

molecule itself does not read extensions/.env. Use task, the wrapper tools/molecule.sh (e.g. tools/molecule.sh test -s dbfs) or load the file into the shell with set -a; . ./.env; set +a before calling molecule directly.

6.2. Taskfile

extensions/Taskfile.yml wraps the molecule commands. The scenario is selected with S= (default: default).

cd extensions
task --list
task pull                      # pull container images before the 1st run
task converge S=dbfs-ol9
task converge S=dbfs-ol9 -- -vv  # arguments after -- are passed to molecule
task converge S=dbfs-ol9 C=false # without reinstalling the collection
task login S=default H=ol9
task destroy S=dbfs-ol9
task collection                # reinstall collection after role changes

task converge and task test install the collection from the repository (task collection) before molecule is started. The dependency step of molecule installs the collection only when the version in galaxy.yml has changed. Without the installation, changes in plugins/ or roles/ are not used. C=false skips the installation.

Several molecule runs may run in parallel, but the installation must not replace the collection while another run uses it (missing files in the running scenario). A lock file (tools/lock.sh, default extensions/.molecule.lock in the checkout, the same for all shells) prevents this:

tools/molecule.sh holds a shared lock during the molecule run (not for login and list).
tools/collection_install.sh needs an exclusive lock and fails immediately, when another run is active. Start the second run with C=false.
tools/regression.sh installs the collection with the exclusive lock and keeps a shared lock for its scenarios.

The tasks are also available from the repository root as task molecule:<task>.

6.3. Test OS roles

Important

This scenario is used, when changes at OS roles should be tested against different OS Distribuitions.
Most of the tests are included in the dbfs scenario!

The execution time is ~10m for a full test.

The default scenario requires no parameter -s!

Start download

cd extensions
molecule test

6.4. Download Patches for `dbfs` stage

Important

Make sure that /vagrant has enough free space for installation media and patches.

set environment variables for support.oracle.com login

export MOS_USERNAME=Username
export MOS_PASSWORD=verySecret

or store them in extensions/.env (ignored by git)

MOS_USERNAME=Username
MOS_PASSWORD='verySecret'

Environment variables have precedence over extensions/.env. Quote the password with ' when it contains special characters of the shell like a blank, $, #, " or ;. Restrict the access to the file with chmod 600 extensions/.env.

Start download

cd extensions
tools/molecule.sh test -s download

molecule test does the full test of the scenario.

6.5. Test Single Instance in Filesystem with scenario `dbfs`

Important

Make sure that the installation media and patches are placed in /vagrant before starting molecule.
Use the scenario download for downloading the Patches from support.oracle.com.

The execution time is ~2h for a full test.

Start download

cd extensions
molecule test -s dbfs

6.6. Create Golden-Images from `dbfs` installation

Important

Make sure that all requirements for scenario dbfs are met.
Executing a full test with molicule test -s golden will destroy an existing installation from molecule converge -s dbfs!

Start Golden-Image creation

cd extensions
molecule test -s golden

6.7. Oracle Autonomous Health Framework (AHF)

The roles of Oracle Autonomous Health Framework (AHF) are tested in three scenarios. The scenario ahf is the AHF Fleet Insights server, dbfs and has-ol9 are clients and upload their data into it.

AHF is downloaded with AutoUpgrade in all of them (extensions/molecule/shared_config/playbooks/ahf_download.yml): latest autoupgrade.jar from Oracle, AHF with target_version: 19, platform: LINUX.X64, patch: AHF into /u01/stage/download in the container. The MOS login is taken from MOS_USERNAME / MOS_PASSWORD or extensions/.env like in the scenario download.

Table 1. Scenarios for AHF
Scenario	Installation media	Test
`ahf`	AHF: AutoUpgrade. AHF Fleet Insights: `AHFFI-LINUX_v*.zip` from `MOLECULE_SW_DIR` or download from My Oracle Support (`molecule_ahffi_file`, `molecule_ahffi_aru` in `host_vars/ahf.yml`, Doc ID 3043060.1)	Containers `ahf` (Fleet Insights, AHF client) and `ahf-db` (repository database, image `gvenzl/oracle-free:23-slim-faststart`, no Ansible), network `molecule-ahf`. prepare: downloads, Oracle Instant Client, schema user `AHFFI` in `FREEPDB1` of `ahf-db` with the grants of `create_db_user.sql` converge: role `ahf_fleet_insights` (certificate with SAN), role `ahf` registered in Fleet Insights idempotence: second run without changes verify: `ahfctl statusahf`, TFA daemon, `ahffi status`, web application on port 5005, registration, tables of `AHFFI`
`has-ol9`	AHF: AutoUpgrade (Java: JDK of the GI-Home)	AHF with Oracle Restart 26ai, ASM and a 19c database, client of Fleet Insights. converge (after `manage_db`): role `ahf` with Compliance scheduler, registration in Fleet Insights verify: `ahfctl statusahf`, TFA daemon, CRS/ASM/RDBMS discovered by TFA (`tfactl print directories`), Compliance autoruns, registration
`dbfs`	AHF: AutoUpgrade (Java: JDK of the DB-Home)	AHF with a 26ai Single Instance, client of Fleet Insights (`extensions/molecule/dbfs/ahf.yml`, after `manage_db`). verify: `ahfctl statusahf`, TFA daemon, registration

The clients use the shared playbooks in extensions/molecule/shared_config/playbooks:

ahf_fleet_connect.yml: connects the container to the network molecule-ahf (docker network connect on the Ansible controller), checks https://ahf:5005/ahfservice and reads the certificate of Fleet Insights from the container ahf.
ahf_client.yml: role ahf and one TFA collection (tfactl diagcollect -last 1h) for the upload. Fleet Insights is optional for dbfs and has-ol9: without the scenario ahf, AHF is installed and verified, the registration is not changed (ahf_fleet_insights_manage: false) and the Fleet Insights checks of verify are skipped (extensions/molecule/shared_config/tasks/ahf_fleet_verify.yml). Only the scenario ahf requires Fleet Insights.

Run the scenarios

cd extensions
task converge S=ahf                # Fleet Insights server first
task verify S=ahf
task converge S=has-ol9 C=false    # clients, C=false while another run is active
task verify S=has-ol9
task converge S=dbfs C=false
task verify S=dbfs

Notes:

The variables are in extensions/molecule/shared_config/inventory/host_vars/ahf.yml (server) and group_vars/all/ahf.yml (URL, network and registration user for all scenarios).
The downloads are executed by prepare of ahf only once after create. They are repeated after destroy.
Fleet Insights needs Internet access for the Oracle JDK and Oracle Instant Client (yum.oracle.com, download.oracle.com).
molecule_ahf_collect: false skips the TFA collection of the clients.
Oracle recommends 16 GB RAM for Fleet Insights. The containers ahf and ahf-db need about 5 GB in idle state.
The data in Fleet Insights is checked in the web interface, there is no automatic check of the uploaded data.
The web interface is published on the host: https://localhost:5005/ahfservice (another port with MOLECULE_AHF_PORT, the mapping is created with the container). For a container created without the mapping: docker run -d --name ahf-proxy --network molecule-ahf -p 5005:5005 docker.io/alpine/socat tcp-listen:5005,fork,reuseaddr tcp-connect:ahf:5005

6.8. Data Guard tests

The side-effect of the scenario dataguard tests the Data Guard configuration created by converge (extensions/molecule/dataguard/side_effect.yml, tasks in extensions/molecule/dataguard/tasks):

Initial state: broker configuration SUCCESS, DB1_SITE1 primary (READ WRITE), DB1_SITE2 physical standby (MOUNTED)
VALIDATE DATABASE (repeated until Ready for Switchover: Yes) and switchover to DB1_SITE2
Roles after the switchover, tempfiles of all temporary tablespaces of the new primary (RMAN DUPLICATE does not copy tempfiles, they are created when the standby is opened as primary)
Switchover back to DB1_SITE1, roles and tempfiles

cd extensions
task side-effect S=dataguard

The tests connect with dgmgrl and sqlplus from dataguard-1 with the connect identifiers of db_standby_config, the SYS password is passed with stdin (no_log). After a failed test the configuration may stay switched over: the next run expects DB1_SITE1 as primary.

6.9. Module tests

The side-effect of the scenarios dbfs, dbfs-ol10 and has-ol9 tests the modules of opitzconsulting.ansible_oracle against an existing database. The tests are shared by the scenarios in extensions/molecule/shared_config/module_tests, only the scenario specific variables are in <scenario>/module_tests.yml.

Scenario Environment

Scenario	Environment
`dbfs`	Oracle AI Database 26ai on filesystem without Grid Infrastructure (CDB `DB1`, PDB `ORCLPDB`)
`dbfs-ol10`	Same as `dbfs` on Oracle Linux 10
`has-ol9`	Oracle Restart 26ai with ASM, Oracle Database 19c in `+DATA` (CDB `DBHAS`, PDB `ORCLPDB1`)

dbfs

Oracle AI Database 26ai on filesystem without Grid Infrastructure (CDB DB1, PDB ORCLPDB)

dbfs-ol10

Same as dbfs on Oracle Linux 10

has-ol9

Oracle Restart 26ai with ASM, Oracle Database 19c in +DATA (CDB DBHAS, PDB ORCLPDB1)

The database is created once with converge, the tests can be repeated as often as needed.

6.9.1. Usage

cd extensions
task converge S=dbfs                                    # once (creates the database)
task module-test                                        # all module tests in dbfs
task module-test M=oracle_user,oracle_role              # selected modules
task module-test L=cx_oracle                            # result file dbfs/module_tests_results/cx_oracle.json
task module-test M=oracle_user C=false                  # without reinstalling the collection
task module-test S=dbfs-ol10                            # all module tests in dbfs-ol10
task module-test S=has-ol9                              # all module tests in has-ol9
task module-test S=has-ol9 M=oracle_asmdg               # selected modules in has-ol9

task module-test installs the collection from the repository (task collection) and runs molecule side-effect -s <scenario> with the environment variables MODULE_TESTS (comma separated list of modules) and MODULE_TESTS_LABEL (name of the result file).

Important

molecule side-effect alone does not install the collection.
Without task collection the tests use the code of the last task converge or task collection.

C=false skips task collection, e.g. for repeated runs when only the tests in module_tests/ have changed (they are read from the repository, not from the installed collection). Changes in plugins/ or roles/ are not used until the collection is installed again.

6.9.2. Structure

<scenario>/side_effect.yml imports shared_config/module_tests/module_tests.yml. The playbook loads vars.yml, known_bugs.yml and <scenario>/module_tests.yml (found with MOLECULE_SCENARIO_DIRECTORY).

Table 2. Variables in `<scenario>/module_tests.yml`
Variable	Purpose
`mt_all_modules`	test files (without `.yml`) of the scenario
`mt_oracle_home`, `mt_oracle_sid`	ORACLE_HOME and ORACLE_SID of the database
`mt_tns_admin`	directory with `tnsnames.ora` and `sqlnet.ora` (wallet)
`mt_cdb`, `mt_pdb`	CDB and PDB for the tests
`mt_datafile_dir`	directory or ASM disk group for explicit data files
`mt_known_bugs_scenario`	known bugs of the scenario
`mt_asm_`, `mt_conn_asm`	ASM instance (only `has-ol9`)

Table 3. Files in `shared_config/module_tests`
File	Purpose
`module_tests.yml`	playbook, imported by `<scenario>/side_effect.yml`
`vars.yml`	connection variants, environment, module selection
`known_bugs.yml`	known bugs in all scenarios (`<check id>: <description>`)
`_module.yml`	run the tests of one module, record an abort as failed check `<module>.aborted`
`_check.yml`	compare a registered module result with the expectation and record it
`_sql.yml`	independent verification with `sqlplus / as sysdba` (or `/ as sysasm`)
`<module>.yml`	tests of one module
`oracle_services.yml`	`oracle_services` without Grid Infrastructure (DBMS_SERVICE, `dbfs`)
`oracle_services_gi.yml`	`oracle_services` with Grid Infrastructure (srvctl, `has-ol9`)
`oracle_asmdg.yml`	`oracle_asmdg` with an own disk group on the spare ASM disks (`has-ol9`)
`oracle_crs_listener.yml`	`oracle_crs_listener` with an own listener `MOLTEST_LSNR` (srvctl, `has-ol9`)

6.9.3. Result of a test run

A failed check does not stop the test run, the play fails at the end (Assert that all checks passed).

An error outside of a check (e.g. a failed setup or cleanup in _sql.yml, a template error in a test) aborts only the tests of this module. It is recorded as failed check <module>.aborted with the failed task and the error message, the other modules are tested anyway. Test objects left behind are removed by the cleanup at the beginning of the next run.

At the end the play shows

Summary per module: a table with the number of checks per module and in total: passed, failed, known (known bug), not repr. (known bug not reproduced) and time (runtime of the tests of the module, also for an aborted module)
Summary of module tests: the result file and the lists of the failed checks, the known bugs and the known bugs not reproduced

The runtime per module is written into <scenario>/module_tests_results/<label>.durations.json (list of module and duration_s) - useful to find slow modules.

Every check is written into <scenario>/module_tests_results/<label>.json on the controller (ignored by git), also after an abort:

Key Content

Key	Content
`id`	id of the check
`module`	test file of the module (entry of `mt_modules`)
`expect`	`changed`, `ok` or `failed`
`actual`	`changed`, `ok`, `failed` or `aborted`
`status`	`passed`, `failed`, `known` (known bug) or `fixed` (known bug not reproduced)
`msg`	message of the module (max. 500 characters)

id

id of the check

module

test file of the module (entry of mt_modules)

expect

changed, ok or failed

actual

changed, ok, failed or aborted

status

passed, failed, known (known bug) or fixed (known bug not reproduced)

msg

message of the module (max. 500 characters)

Comparison before/after a change

task module-test L=before
task module-test L=after
diff molecule/dbfs/module_tests_results/before.json molecule/dbfs/module_tests_results/after.json

6.9.4. Known bugs

known_bugs.yml (mt_known_bugs_common) contains the ids of checks that fail because of a known bug in a module in all scenarios. Bugs that depend on the environment (database version, Grid Infrastructure) are in mt_known_bugs_scenario in <scenario>/module_tests.yml.

They are reported as KNOWN BUG and do not fail the test run. A check from these lists that passes is reported as KNOWN BUGS NOT REPRODUCED ⇒ remove the entry after the bug has been fixed.

6.9.5. Conventions for a module test

File module_tests/<module>.yml, added to mt_all_modules in <scenario>/module_tests.yml.
No scenario specific values in the tests: use mt_cdb, mt_pdb, mt_oracle_home, mt_datafile_dir etc.
All test objects start with MOLTEST_ (common objects in the CDB: C##MOLTEST_).
The test is repeatable and leaves the database unchanged: remove the objects at the beginning (cleanup, result ignored) and at the end. Changed parameters, AWR settings etc. are restored.

Every module call has register: _res and ignore_errors: true, followed by _check.yml with a unique id <module>.<cdb|pdb|asm>.<step> and the expectation:

- name: oracle_role | create
  opitzconsulting.ansible_oracle.oracle_role:
    role: MOLTEST_ROLE
  register: _res
  ignore_errors: true
- name: oracle_role | check create
  ansible.builtin.include_tasks: _check.yml
  vars: {mt_id: oracle_role.pdb.create, mt_res: "{{ _res }}", mt_expect: changed}

Typical steps: create (changed), same call again (ok), check mode if supported (changed, verify with sqlplus that nothing happened), change (changed), again (ok), verify with _sql.yml, drop (changed), drop again (ok), error case (failed).
The expectation is the documented behaviour (DOCUMENTATION of the module). If the module behaves differently because of a bug, the check keeps the documented expectation and the id goes into known_bugs.yml (all scenarios) or mt_known_bugs_scenario (only one scenario).
Connection: hostname, port, service_name, user, password and mode come from module_defaults in module_tests.yml. Default is mt_conn_pdb (system in mt_pdb). Switch it with vars: {mt_conn: "{{ mt_conn_cdb }}"} on the task. Variants in vars.yml: mt_conn_cdb, mt_conn_pdb, mt_conn_cdb_sysdba, mt_conn_pdb_sysdba, mt_conn_pdb_wallet, mt_conn_pdb_wrong_password. Every module test uses the wallet and sysdba at least once.

Verification with sqlplus (independent of the Python driver), in the ASM instance with mt_sql_asm: true (sqlplus / as sysasm):

- name: oracle_role | verify create
  ansible.builtin.include_tasks: _sql.yml
  vars:
    mt_sql_container: "{{ mt_pdb }}"
    mt_sql: "select count(*) from dba_roles where role = 'MOLTEST_ROLE';"
    mt_sql_expect: "1"
    mt_sql_id: oracle_role.pdb.create_sql

Not tested: oracle_asmvol (no ADVM/ACFS in a container), oracle_ldapuser (LDAP), oracle_opatch, oracle_gi_facts, oracle_sqldba (no python-oracledb).

6.10. Controller-only tests of oracle_databases

task test:oracle_databases (in extensions/) tests combinations of attributes in oracle_databases on localhost without container, VM or database (about 20 seconds):

asserts of orasw_meta (assert_oracle_databases.yml): omf, is_container, storage_type, db_create_file_dest / db_recovery_file_dest in init_parameters, removed datafile_dest / recoveryfile_dest
destinations for dbca from oradb_manage_db/vars/main.yml
values in the dbca response file (19.3.0.0, 26.1.0.0), e.g. createAsContainerDatabase

The test cases are in extensions/tests/oracle_databases/cases.yml (expect: ok|failed, optional warn, dest and rsp). The playbook fails at the end with a list of all failed checks.

Run without Taskfile

ansible-playbook -i localhost, extensions/tests/oracle_databases/test_oracle_databases.yml

6.11. Regression tests

task molecule:regression:<preset> runs several scenarios with a report per run (extensions/Taskfile.regression.yml, extensions/tools/regression.sh).

The collection is installed once before all scenarios.
The scenarios run in parallel (J, default 6), the steps of a scenario one after the other.
A failed create or converge skips the remaining steps of the scenario.

Table 4. Presets
Task	Steps
`molecule:regression:final`	destroy, converge, module-test
`molecule:regression:full`	destroy, converge, idempotence, module-test
`molecule:regression:retest`	module-test in the existing containers
`molecule:regression:idempotence`	idempotence, module-test in the existing containers
`molecule:regression:run T=<steps>`	free combination of destroy, create, converge, idempotence, module-test, verify
`molecule:regression:report RUN=<run-id>`	create and show `summary.md` (default: last run)
`molecule:regression:list`	last runs with the number of failed steps

Variables: S scenarios (comma separated, default dbfs,dbfs-sh,has-ol9,dataguard), J scenarios in parallel, M module tests (MODULE_TESTS).

Every run creates extensions/test-runs/<date>-<time>_<commit>[-dirty]/ (not in git), extensions/test-runs/latest points to the last run:

File Content

File	Content
`run.env`	commit, branch, parameters, start and end
`git-status.txt`	uncommitted changes of the tested code
`steps.jsonl`	one JSON line per finished step: scenario, step, rc, duration, logfile
`<scenario>-<step>.log`	output of molecule without colors
`module_tests/<scenario>.json`	result of the module tests (`MODULE_TESTS_LABEL=<run-id>`)
`module_durations/<scenario>.json`	runtime per module of the module tests
`summary.md`	report: scenario x step, module tests, runtime per module (slowest first), changed tasks of idempotence, errors of failed steps

run.env

commit, branch, parameters, start and end

git-status.txt

uncommitted changes of the tested code

steps.jsonl

one JSON line per finished step: scenario, step, rc, duration, logfile

<scenario>-<step>.log

output of molecule without colors

module_tests/<scenario>.json

result of the module tests (MODULE_TESTS_LABEL=<run-id>)

module_durations/<scenario>.json

runtime per module of the module tests

summary.md

report: scenario x step, module tests, runtime per module (slowest first), changed tasks of idempotence, errors of failed steps

summary.md is updated after every step and can be read while the run is in progress. Failures with fatal: are shown first, failures of loop items (failed: [host] (item=…)) are often rescued negative tests.

6.12. Run molecule tests with Claude Code

tools/claude/skills/aotest contains the skill /aotest for Claude Code. It runs the molecule actions of extensions/Taskfile.yml inside nix-shell and reports the result.

Install the skill like /pr (see Create Pull-Requests with Claude Code):

mkdir -p ~/.claude/skills
ln -sfn "$(pwd)/tools/claude/skills/aotest" ~/.claude/skills/aotest

Usage

/aotest [S=<scenario>] [T=full|destroy|converge|module[,...]] [M=<module>[,<module>...]]

Argument Default Description

Argument	Default	Description
`S=<scenario>`	`dbfs`	Scenario in `extensions/molecule`.
`T=<action>[,…]`	`module`	Actions in the given order: `full` (`task test`), `destroy`, `converge`, `module` (`task module-test`).
`M=<module>[,…]`	all modules	Modules for `T=module`, see Module tests.

S=<scenario>

dbfs

Scenario in extensions/molecule.

T=<action>[,…]

module

Actions in the given order: full (task test), destroy, converge, module (task module-test).

M=<module>[,…]

all modules

Modules for T=module, see Module tests.

Examples

/aotest                                   # module tests in dbfs
/aotest S=has-ol9 M=oracle_asmdg
/aotest S=dbfs T=destroy,converge,module
/aotest S=dbfs-ol10 T=full

Scenario, actions and modules are checked before the start.
Each action runs in the background with its own log file. A failed action stops the following actions.
T=module on a scenario without running instances is not started, the skill proposes T=converge,module.
The report contains the PLAY RECAP, the summary of the module tests and the failed checks with message. The skill does not change code or tests.

Important

Invoking /aotest allows Claude to run the requested actions, including destroy and converge, without further confirmation.

6.13. Oracle XE (not used at the moment!)

Important

This chapter was designed for testing the oradb_managed_-Roles.
It is not clear if this will be added or not.

Docker pull Oracle XE

Normal SSO-Login from Oracle is needed for login to docker registry.

docker login container-registry.oracle.com
docker pull container-registry.oracle.com/database/express:21.3.0-xe

Start Oracle Container for test

docker run -d --name oracle-xe \
  -e ORACLE_PWD=Oracle_123 \
  -p 1521:1521 \
  container-registry.oracle.com/database/express:21.3.0-xe

2 - Website

1. How to develop the documentation page?

The documentation is build with hugo and Docsy as template.
There is a container for building or starting a hugo server for easy development of the documentation.

The Setups between Docker or Podman is a little bit different.

2. Where is the content?

The pages of the documentation are not copied into website/. hugo reads them directly from the repository:

Source Page

doc/.adoc, doc/guides/.adoc, doc/*.md

Guides, features and development

roles/*/README.md

One page per role (generated by ansible-doctor)

website/content/

Start page, About and the introduction of each section (_index.md)

website/hugo.yaml mounts doc/ and roles/ below assets/. The content adapter website/content/docs/_content.gotmpl creates the pages from them:

The first heading of a file is used as the page title, unless a title is set in $settings of the content adapter.
Relative links (AsciiDoc link: macro or Markdown link) are rewritten to the pages of the website. Links to files without a page point to GitHub.
The "Edit this page" link points to the source file below doc/ or roles/.
"Last modified" shows the last commit of the source file. scripts/gitinfo.py writes it to website/data/gitinfo.json before each build (hugo would show the commit of the content adapter instead).
A new file in doc/guides/ appears automatically. New files in other directories need a rule in $rules of the content adapter.

Important

Do not use include:: in files below doc/. hugo passes the content to asciidoctor by stdin and GitHub does not render includes either. Use a link instead.

3. Tasks

website/Taskfile.yml contains the commands for building and testing the page. task is part of the nix-shell of ansible-oracle.

cd ansible-oracle/website
task                # list all tasks
task serve          # live server on http://localhost:1313
task build          # build the page into public/
task test           # build + check internal links
task test:a11y      # build + accessibility check (WCAG 2 AA) of all pages
task publish        # build + commit in the clone of the website repository
task publish:push   # build + commit + push => GitHub Pages publishes the website
task publish:pr     # build + commit + pull request in the website repository

The tasks can be started from the root of the repository with prefix website:, e.g. task website:test.

4. Publishing

The website is published with GitHub Pages from the directory docs/ in the branch main of Rendanic/ansible-oracle-website.

The publish tasks call tools/release/build_website.sh:

The sources in website/, doc/, roles/ and docker/hugo/ must be committed.
The legal notice must be published (no draft: true).
The website repository is cloned to ~/.cache/ansible-oracle-website with gh.
docs/ is replaced by the new build, CNAME is kept and .nojekyll is added (GitHub Pages ignores directories with a leading _ like _print/ without it).

Options of the script are passed after --, e.g. task publish — -a for an additional accessibility check or task publish — -f to ignore uncommitted changes and a missing legal notice (for tests only).

Podman is used with task build COMPOSE=podman-compose COMPOSE_USER=root.

The checks run in containers:

check:links uses lychee for all links in public/.
check:a11y uses pa11y-ci with the runners axe and HTML_CodeSniffer for all pages in sitemap.xml. The settings are in website/.pa11yci.json. The axe rule color-contrast is ignored: axe cannot determine the background below the fixed navbar and sidebar and reports hundreds of false positives. The contrast is still checked by HTML_CodeSniffer.

5. Colors and contrast

The whole site uses one polarity: dark text on a light background. Docsy’s dark navbar, cover blocks and footer are replaced in

website/layouts/partials/navbar.html (copy of Docsy with data-bs-theme="light")
website/assets/scss/_styles_project.scss (navbar, footer, buttons, links and Prism token colors)

All colors have a contrast ratio of at least 4.5:1 (WCAG 2 AA). Use blocks/lead or blocks/section with color="white" on new pages, not blocks/cover. Code blocks are highlighted by Prism (light theme), Chroma is disabled for Markdown code fences in hugo.yaml.

6. Updates of hugo and Docsy

File Version Updated by

docker/hugo/Dockerfile

hugo and asciidoctor (base image floryn90/hugo)

renovate

website/hugo.yaml

module.hugoVersion.min

manual

website/go.mod, website/go.sum

Docsy (github.com/google/docsy)

renovate (group docsy)

website/Taskfile.yml

lychee, puppeteer, pa11y-ci, http-server (# renovate: comments)

renovate

The image of the hugo container is tagged with the tag of the base image, e.g. rendanic/hugo:0.169.0-ext-ubuntu. A new version in the Dockerfile builds a new image automatically with the next task build.

renovate updates only website/go.mod and website/go.sum. go from shell.nix is needed for go.sum. The steps below are still manual, the Pull-Request of renovate (group docsy) lists them as a note. There is no github workflow which builds the website, so a Docsy Pull-Request is green even when the website is broken.

Steps for an update of Docsy:

Read the release notes of Docsy (minimum version of hugo, breaking changes). Docsy 0.16 and later is a different Hugo module (github.com/docsy/docsy/theme) with Bootstrap and Font Awesome from npm. renovate.json allows only versions below 0.16.
Update website/go.mod and website/go.sum. Run task clean:all afterwards: an old module in website/.cache/modules can remain as empty directories and is not downloaded again (error File to import not found or unknown output format).
task docsy:diff shows the differences between the copied templates in website/layouts/ and the new Docsy version. Apply the changes of Docsy to the copies.
Check website/assets/scss/_styles_project.scss: it uses the Docsy classes .td-navbar, .td-footer, .td-box and .td-content.
task test:all and a look at the start page, a guide and a role page.

Note	`security.exec.osEnv` in `website/hugo.yaml` passes `GEM_HOME` to `asciidoctor`. Without it, `asciidoctor` does not find its gem in the base image.

Note	Since hugo 0.170, the SCSS of Docsy is compiled with Dart Sass (`sass` in the base image), libsass was removed. `security.exec.allow` in `website/hugo.yaml` must contain `^sass$`. The deprecation warnings of Dart Sass come from Docsy and Bootstrap.

7. Container Setup

The setup of Docker or Podman is described in the Development Guide.

The Dockerfile and docker-compose.yaml of the hugo container are in docker/hugo/. The image is built automatically on the first task build or with task hugo:build in docker/.

8. hugo with Docker

8.1. hugo live server

Important

This only works when current uid in shell is same as ubuntu(1000) inside the container.

cd ansible-oracle/docker/hugo
docker compose run --rm -u ubuntu site -s /src/website server

8.2. Building the public page

Important

This only works when current uid in shell is same as ubuntu(1000) inside the container.

cd ansible-oracle/docker/hugo
docker compose run --rm -u ubuntu site -s /src/website

9. hugo with Podman

9.1. hugo live server

The default user inside the container is hugo with a different uid then the nonroot user of the podman Container.
This results in a permission denied when the uid of nonroot podman user is different then the uid of hugo inside the container while reading volume data. Starting with --user=root starts all processes in the same namespace as the nonroot user which allows access to the volume data.

Start hugo server with Podman

cd ansible-oracle/docker/hugo
~/venv/podman/bin/podman-compose run --rm --user=root site -s /src/website server --bind 0.0.0.0

9.2. Building the public page

cd ansible-oracle/docker/hugo
~/venv/podman/bin/podman-compose run --rm --user=root site -s /src/website

Development

1 - How to develop

1. How to develop in ansible-oracle?

1.1. Create Fork on github and contribute to the project

1.2. Development environment

1.2.1. Option 1: nix-shell (recommended)

1.2.1.1. What is nix-shell?

1.2.1.2. Why nix-shell instead of containers?

1.2.1.3. Install Nix

1.2.1.4. Using nix-shell

1.2.1.5. Update the venvs

1.2.2. Option 2: local Ansible-Controller

1.2.2.1. venv for ansible-oracle

1.2.2.2. venv for ansible-doctor

1.2.2.3. Ansible-Container

1.3. Working with the tools

1.3.1. pre-commit

1.3.2. ansible-doctor

1.3.3. Start Playbook

2. Pull-Requests

2.1. Important Information

2.2. Create Pull-Requests with Claude Code

2.2.1. Setup

2.2.2. Usage

2.2.3. Behavior

3. Create new Releases in ansible-oracle

3.1. Install antsibull-changelog

3.2. Working with antsibull-changelog

3.3. Add PR references to fragments

3.4. Collection version in requirements.yml

3.5. Creating new releases

3.5.1. Release branches

3.5.2. Release-Pull-Request (-m pr)

3.5.3. Draft release (-m release)

3.5.4. Update the website (-m docs)

3.5.5. Sync fork and deploy to Ansible Galaxy (-m galaxy)

3.5.6. Tasks

3.5.7. Options

4. Check dependencies with renovate

4.1. Local checks

4.1.1. ocenv from oracle-scripts

4.1.2. Oracle Release Updates for molecule

4.2. Setup gh for renovate

4.3. Run renovate against your fork

4.4. Run renovate against upstream

5. Update Vagrantbox on vagrantcloud for examples

5.1. Create new Box

5.2. Upload new Version

6. ansible molecule

6.1. Directory for installation media and patches

6.2. Taskfile

6.3. Test OS roles

6.4. Download Patches for dbfs stage

6.5. Test Single Instance in Filesystem with scenario dbfs

6.6. Create Golden-Images from dbfs installation

6.7. Oracle Autonomous Health Framework (AHF)

6.8. Data Guard tests

6.9. Module tests

6.9.1. Usage

6.9.2. Structure

6.9.3. Result of a test run

6.9.4. Known bugs

6.9.5. Conventions for a module test

6.10. Controller-only tests of oracle_databases

6.11. Regression tests

6.12. Run molecule tests with Claude Code

6.13. Oracle XE (not used at the moment!)

2 - Website

1. How to develop the documentation page?

2. Where is the content?

3. Tasks

4. Publishing

5. Colors and contrast

6. Updates of hugo and Docsy

7. Container Setup

8. hugo with Docker

8.1. hugo live server

8.2. Building the public page

9. hugo with Podman

9.1. hugo live server

1. How to develop in `ansible-oracle`?

3. Create new Releases in `ansible-oracle`

3.5.2. Release-Pull-Request (`-m pr`)

3.5.3. Draft release (`-m release`)

3.5.4. Update the website (`-m docs`)

3.5.5. Sync fork and deploy to Ansible Galaxy (`-m galaxy`)

6.4. Download Patches for `dbfs` stage

6.5. Test Single Instance in Filesystem with scenario `dbfs`

6.6. Create Golden-Images from `dbfs` installation