# Usage

Usage of `run-ci` command and other tools provided by ocs-ci repository.

# Usage for run-ci tool

For full usage run: `run-ci --help`

## Required configuration

For other prerequisites and setup instructions, see [Getting Started](./getting_started.md) doc.

* **AWS credentials** - if you have AWS already configured by `aws configure`,
    see:
    [AWS doc](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html)
    there shouldn't be need of any additional configuration for AWS. If not,
    please make sure you configure your AWS credentials via `aws configure`
    command, or you have proper configuration done in `~/.aws` folder which is
    usually generated by mentioned command.

## Required arguments

There are a few arguments that are required ocs test execution:

* `--cluster-path <path>` - path to the directory which will
    contain all the installation/authentication information about the cluster.
    If the information given can not be used to access the cluster then
    test execution will fail. If you wish to deploy a new cluster, give
    a path to a new directory and also use the `--deploy` argument.

## Useful arguments

Some non-required arguments that we end up using a lot. You can use
`run-ci  --help` to see all the parameters and description which you can pass
to the pytest.

* `--capture=no` - when using pdb or ipdb you have to turn of capture mode
    for pytest.
* `-m "tier1 and ecosystem"` - This will select just tests marked with
    tier1 and ecosystem marks.

### Subcommand:
* `multicluster <int>` - to be used if multiple clusters needs to be handled by ocs-ci, this subcommand is useful in the
   DR scenario where multiple cluster contexts needs to be handled by the framework.
   For more information on the usage check examples section and `run-ci multicluster --help` and
   [Multi-cluster test example](./multicluster_test_example.md) doc.


### Additional arguments:

* `--cluster-name <name>` - name of cluster (always required for deployment) must be 5-17 characters long.
* `--ocs-version` - version of OCS to be used (e.g. 4.2 or 4.3). If not
    specified, the default from ocs-ci will be used. If `--ocs-registry-image` is passed
    then `--ocs-version` parameter has higher priority which allow us to install previous version
    from CSV of the next build (e.g. we can install 4.3 from 4.4 build csv).
* `--upgrade-ocs-version` - version of OCS to be used for upgrade (e.g. 4.2 or 4.3). If not specified, the default from ocs-ci will be used.
* `--ocp-version` - OCP version to be used for deployment. This version will
    be used for load file from conf/ocp_version/ocp-VERSION-config.yaml.
    You can use for example those values:
  * `4.2`: for nightly 4.2 OCP build
  * `4.2-ga`: for latest GAed 4.2 OCP build from stable channel
  * `4.2-ga-minus1`: for latest GAed 4.2 build (stable channel) - 1
* `--ocp-installer-version` - Specific OCP installer version to be used for deployment.
  This option will generally be used for non-GA or nightly builds. (e.g. 4.5.5).
  This option will overwrite any values set via --ocp-version
* `--ocs-registry-image` - ocs registry image to be used for deployment
    (e.g quay.io/rhceph-dev/ocs-olm-operator:latest-4.2). In case this parameter
    is passed, the version is parsed from the registry image name but only
    if no version is passed via --ocs-version parameter.
* `--upgrade-ocs-registry-image` - ocs registry image to be used for upgrade (e.g quay.io/rhceph-dev/ocs-olm-operator:latest-4.3)
* `--ocsci-conf` - with this configuration parameter you can overwrite the
    default OCS-CI config parameters defined in
    [default config](https://github.com/red-hat-storage/ocs-ci/tree/master/ocs_ci/framework/conf/default_config.yaml).
    This is the repeatable parameter and you can use it multiple times for
    adding multiple config files. (The last one overwrites previous config!)
* `--deploy` - if this is given and a cluster can not be accessed from the
    provided `--cluster-path` then a new test cluster will be deployed.
* `--live-deploy` - OCS-CI will use live content to deploy OCS and will also
  update OCS/ODF must gather image to the GAed one.
* `--teardown` - if this is given the testing cluster will be destroyed after
    the test have completed, regardless of if the tests passed or failed.
* `--html` - to generate html reports of the test run
* `--self-contained-html` - creates self-contained html file containing all
   necessary styles and scripts
* `--email` - to send the email reports of the test run which was generated
   by --html option. MUST specify --html to send email reports.
* `--squad-analysis` - Include Squad Analysis to email report. Applicable only
    in combination with --email option.
* `--bugzilla` - allows you to skip tests with unresolved bug (test case
  needs to be properly decorated as you can see in
  [documentation](./writing_tests.md)). You will also need to
  create configuration file for `pytest_marker_bugzilla` plugin. See the
  [plugin documentation](https://github.com/eanxgeek/pytest_marker_bugzilla)
  or example of `/etc/bugzilla.cfg` file:
  ```bash
  [DEFAULT]
  bugzilla_url = https://bugzilla.redhat.com/xmlrpc.cgi
  bugzilla_api_key = <API_KEY>
  ```
* `--collect-logs` - to collect OCS logs for failed test cases.
* `--collect-logs-on-success-run` - Collect must gather logs at the end of the
   execution (also when no failure in the tests)
* `--io-in-bg` - If passed, IO will be running in the test background.
* `--io-load` - IOs throughput target percentage. The value should be
   between 0 to 100. If not provided, the default is 30 (30%)
* `--log-cluster-utilization` - If passed, metrics of the cluster utilization will be
   printed every 20 seconds
* `--csv-change` - Allows changes in the OCS CSV. For example, usage of custom image,
   like MCG or RHCS. The format should be:
   <pattern_to_replace_from::pattern_to_replace_to>, while '::' is the delimiter.
   This argument is repeatable and you can use it multiple times to change multiple images.
* `--dev-mode` - Runs in development mode. Skip the checks like collecting
   cluster versions, collection ocs versions, health checks etc.
* `--ceph-debug` - Deploy with Ceph in debug log level. This option is available starting OCS 4.7
* `--re-trigger-failed-tests` - Path to the xunit file for xml junit report from the
    previous execution. If the file is provided, the execution will remove all the test cases
    which passed and will run only those test cases which were skipped / failed / or had error
    in the provided report.
* `--install-lvmo` - Deploy LVMCluster, will skip ODF deployment.
* `--lvmo-disks` - Number of disks to add to SNO deployment.
* `--lvmo-disks-size` - Size of disks to add to SNO deployment.
* `--disable-environment-checker` - Disable the leftover checks in existing flow.
* `--resource-checker` - This will identify the leftover which was created by test cases. This is
    similar to environment-checker, only difference is resource-checker will track the resources
    created during test case run whereas environment-checker will track all resources in
    cluster irrespective of who created.
* `--kubeconfig` - Location of kubeconfig.

## Examples

Deployment and teardown of the test cluster can be done automatically with
the `--deploy` and `--teardown` arguments. The `--cluster-path` must always be
provided and if given without the `--deploy` argument, it must contain information
that can be used to access an existing cluster.

> In case you lost your cluster dir, the destroy can be done with
> `ci-cleanup` cli that should be invoked with cluster-tag from AWS.

### Deployment of cluster

If you would like to deploy new cluster you can run following command:

```bash
run-ci -m deployment \
    --ocsci-conf conf/path_to_config_to_be_used.yaml \
    --cluster-name kerberos_ID-ocs-deployment \
    --cluster-path /home/my_user/my-ocs-dir \
    --deploy \
    tests/
```

Of course you can omit `--cluster-name` if you would like to use default
values.

> **_NOTE:_** You need to specify `tests/` directory as a test path even for
> deployment.

Note that during deployment, openshift command line tools like `oc` and
`openshift-install` are installed into [`bin` directory of the
repository](https://github.com/red-hat-storage/ocs-ci/tree/master/bin). These tools are then available to both deployment and test
code because `run-ci` wrapper includes the `bin` directory into `PATH`
environment variable.

#### Deployment configurations

- *encryption at REST* - to enable encryption at REST use this configuration file:
    conf/ocsci/encryption_at_rest.yaml
- *FIPS* - to enable FIPS, use this configuration file:
    conf/ocsci/fips.yaml

#### Deployment of cluster on vSphere Platform

terraform and jq is needed for deployment of OCS on vSphere platform.

Change the [vsphere_upi_vars.yaml.example](https://github.com/red-hat-storage/ocs-ci/tree/master/conf/ocsci/vsphere_upi_vars.yaml.example) to vsphere_upi_vars.yaml and update the values accordingly.
Check for vsphere_upi_vars.yaml.skeleton(../conf/ocsci/vsphere_upi_vars.yaml.skeleton) for more options.

After filling all the required parameters, run the following command:
```bash
run-ci -m deployment --ocsci-conf conf/ocsci/vsphere_upi.yaml \
    --cluster-name kerberos_ID-ocs-deployment \
    --cluster-path /home/my_user/my-ocs-dir tests/ \
    --deploy
```

#### Deployment via UI

To enable UI deployment please pass `--ocsci-conf conf/ocsci/ui_deployment.yaml`
In addition, the function `ui_deployment_conditions` checks if the deployment type, OCP/OCS version or platform is supported for installation via UI, and if not it will back off and deploy OCS via CLI.
At the moment, UI deployment is possible for the following platforms:
- AWS
- vSphere

#### Running tests on deployed environment

```bash
run-ci -m "tier1 and manage" \
    --cluster-name kerberos_ID-ocs-deployment \
    --cluster-path /home/my_user/my-ocs-dir tests/
 ```

#### Running tests on deployed environment and sending reports

If you would like to send the test run results to email you can run
following command:
```bash

run-ci tests/ \
    --cluster-name kerberos_ID-ocs-deployment \
    --cluster-path /home/my_user/my-ocs-dir \
    --html=report.html --self-contained-html \
    --email=<emailid>
 ```

If you want to send reports to multiple email ID's, use comma separated
email ID's like below
```bash

run-ci tests/ \
    --cluster-name kerberos_ID-ocs-deployment \
    --cluster-path /home/my_user/my-ocs-dir \
    --html=report.html --self-contained-html \
    --email=<emailid1>,<emailid2>,<emailid3>
 ```


#### Running tests on multicluster environment

If you would like to run multicluster environment tests and deployments, use `multicluster` subcommand for run-ci.

##### example 1:
```bash
run-ci multicluster 2 \
    tests/ -m tier1 \
    --cluster1 \
    --cluster-name test_cluster1 --cluster-path test_cluster1_path \
    --ocsci-conf /path/to/cluster1_conf1 --ocsci-conf /path/to/cluster1_conf2 \
    --cluster2 \
    --cluster-name test_cluster2 --cluster-path test_cluster2_path \
    --ocsci-conf /path/to/cluster2_conf1 --ocsci-conf /path/to/cluster2_conf2
```
`multicluster` cluster subcommand is slightly different from usual CLI used in run-ci.
`multicluster` subcommand should be followed by an integer which indicates how many clusters we want to handle with this run, which
will be followed by common arguments like pytest directory path, pytest markers, `deploy`, `teardown` etc. All the common options should appear before
any cluster specific options in the CLI.
Towards the end of CLI, user needs to pass individual cluster specific arguments with cluster sequence numbers like `--cluster1 <cluster1 options>`, `--cluster2 <cluster2 options>` so on.
Cluster specific options will be passed within `--cluster<n>` argument boundary i.e. anything starting from `--cluster1` and before `--cluster2` will be considered
as arguments of `cluster1`.

##### example 2:
Passing common arguments to cluster:
```bash
run-ci multicluster 2 \
    tests/ -m tier1 --ocsci-conf common-conf.yaml \
    --cluster1 --cluster-name test_cluster1 --cluster-path test_cluster1_path \
    --ocsci-conf /path/to/cluster1_conf1 --ocsci-conf /path/to/cluster1_conf2 \
    --cluster2 \
    --cluster-name test_cluster2 --cluster-path test_cluster2_path \
    --ocsci-conf /path/to/cluster2_conf1 --ocsci-conf /path/to/cluster2_conf2
```
In the above example `common-conf.yaml` configuration will be applied on both the clusters.


#### Running tests with background IO load

If you would like to run tests with IO load of 75% of the cluster throughput limit,
in the tests background, append these arguments to the `run-ci` command:
`--io-in-bg --io-load 75`

If you would like metrics of cluster utilization to be logged, every 10 seconds, during
the test execution, append the `--log-cluster-utilization` argument to the `run-ci` command

#### Destroy of cluster

If you would like to destroy existing cluster you can run following command:
```bash
run-ci -m deployment \
    --cluster-name kerberos_ID-ocs-deployment \
    --cluster-path /home/my_user/my-ocs-dir tests/ \
    --teardown
 ```

# usage for getting various ocs image versions

for full help run: `report-version --help`

Run `report-version --cluster-path /my-cluster/path --log-level INFO` to
get information on various image version deployed by ocs-ci

# usage for cleanup

## AWS
  This should be used only when your cluster-dir is accidentally deleted
  `ci-cleanup [-h] --cluster CLUSTER`
  CLUSTER points to cluster tag eg: mycluster-ocs-ci-jlgzn , without additional
   -master or -worker. This can be found by logging into AWS, Selecting the VM
   and clicking on Tags.

## vSphere
  This should be used only when your cluster-dir is accidentally deleted
  ```bash
  vsphere-cleanup [-h] --cluster-name CLUSTER_NAME --ocsci-conf VSPHERE_CONF
  ```
  e.g:

  `vsphere-cleanup --cluster-name mycluster-oct12
                   --ocsci-conf ~/vSphere-DC-CP_VC1.yaml`

# usage to shutdown AWS nodes when nodes are not in use
  Following cli can be used when you want to shutdown the cluster
  `ci-pause --cluster-path /home-dir/cluster-path --action [stop|start]`
  action can be start or stop that performs the required action on all the nodes
  in the cluster

# Usage for ocs-build tool

Use `ocs-build --help` for printing the options.
This tool returns latest OCS build tag which is available and by default will
be used (output example: 4.7.0-353.ci).

If --image parameter is provided, the whole image will be returned (output example:
quay.io/rhceph-dev/ocs-registry:4.7.0-353.ci)

You can also ask for build of specific OCS version using `--ocs-version 4.8`
argument to get tag for 4.8 version (if no version provided, the OCS-CI default
will be used).

The quay access_token is required to be set in data/auth.yaml file.

# Usage for rosa-ocp-version tool

For full usage run: `rosa-ocp-version --help`:
```bash
$ rosa-ocp-version -h
usage: rosa-ocp-version [-h] [--debug] [--get-available-versions] [--check-available-version OCP_VERSION] --ocsci-conf OCSCI_CONF

Get information about available OCP versions from ROSA

options:
  -h, --help            show this help message and exit
  --debug, -d           Print logging messages (mainly useful for debugging).
  --get-available-versions, -a
                        Get all available OCP versions from ROSA
  --check-available-version OCP_VERSION, -c OCP_VERSION
                        Check if provided OCP version is available in ROSA, if available, returns latest z-stream version
  --ocsci-conf OCSCI_CONF
                        OCM Credentials configuration file in yaml format. Example file: --- AUTH: openshiftdedicated: token: '<TOKEN>'
```

## Get list of all available OCP versions in ROSA

```bash
$ rosa-ocp-version --ocsci-conf path/to/credentials-file.yaml --get-available-versions
4.17.14
4.17.12
4.17.11
4.17.10
4.17.9
4.17.8
4.17.7
4.17.6
4.17.5
4.17.4
...
```

## check if provided OCP version is available in ROSA

```bash
$ rosa-ocp-version --ocsci-conf path/to/credentials-file.yaml --check-available-version 4.17
4.17.14
$ echo $?
0

$ rosa-ocp-version --ocsci-conf path/to/credentials-file.yaml --check-available-version 4.18
Version 4.18 not available in ROSA.
$ echo $?
255
```