ocs_ci.utility package

Subpackages

• ocs_ci.utility.perf_dash package
  • Submodules
  • ocs_ci.utility.perf_dash.dashboard_api module
    • PerfDash
      • PerfDash.add_results()
      • PerfDash.cleanup()
      • PerfDash.connect()
      • PerfDash.get_build_id()
      • PerfDash.get_id_by_name()
      • PerfDash.get_name_by_id()
      • PerfDash.get_next_sample()
      • PerfDash.get_platform_id()
      • PerfDash.get_platform_name()
      • PerfDash.get_results()
      • PerfDash.get_test_id()
      • PerfDash.get_test_name()
      • PerfDash.get_topology_id()
      • PerfDash.get_topology_name()
      • PerfDash.get_version_builds()
      • PerfDash.get_version_id()
      • PerfDash.get_version_name()
      • PerfDash.insert_build()
      • PerfDash.insert_single_value()
  • Module contents
• ocs_ci.utility.spreadsheet package
  • Submodules
  • ocs_ci.utility.spreadsheet.spreadsheet_api module
    • GoogleSpreadSheetAPI
      • GoogleSpreadSheetAPI.get_cell_value()
      • GoogleSpreadSheetAPI.insert_row()
      • GoogleSpreadSheetAPI.print_sheet()
      • GoogleSpreadSheetAPI.update_sheet()
  • Module contents
• ocs_ci.utility.tests package
  • Submodules
  • ocs_ci.utility.tests.test_azure_utils module
    • test_azureutil_init_noresourcegroup()
    • test_azureutil_init_resourcegroup()
    • test_azureutil_init_resourcegroup_config()
  • ocs_ci.utility.tests.test_deployment_utils module
    • test_get_cluster_prefix()
  • ocs_ci.utility.tests.test_gcp module
    • test_gcputil_init_region()
    • test_gcputil_init_region_config()
  • ocs_ci.utility.tests.test_prometheus module
    • query_range_result_bad_last_90s()
    • query_range_result_delay_60s()
    • query_range_result_ok()
    • query_range_result_single_error()
    • test_check_query_range_result_exp_delay()
    • test_check_query_range_result_exp_good_time()
    • test_check_query_range_result_exp_metric_num()
    • test_check_query_range_result_null()
    • test_check_query_range_result_simple()
    • test_check_query_range_result_simple_fail()
    • test_check_query_range_result_single_error()
  • ocs_ci.utility.tests.test_rgwutils module
    • test_get_rgw_count()
  • ocs_ci.utility.tests.test_utils module
    • A
    • B
    • test_filter_unrepresentable_values()
    • test_get_attr_chain()
    • test_get_empty_attr()
    • test_get_none_obj_attr()
    • test_get_nonexistent_attr_chain()
    • test_mask_secret_nomatch()
    • test_mask_secret_nosecrets()
    • test_mask_secret_null()
    • test_mask_secret_simple_positive()
    • test_run_cmd_simple_negative()
    • test_run_cmd_simple_negative_ignoreerror()
    • test_run_cmd_simple_negative_with_secrets()
    • test_run_cmd_simple_positive()
    • test_run_cmd_simple_positive_with_secrets()
  • ocs_ci.utility.tests.test_utils_timeout_sampler module
    • test_ti_func_values()
    • test_ts_func_exception()
    • test_ts_null()
    • test_ts_one_iteration()
    • test_ts_one_iteration_big_func_runtime()
    • test_ts_one_iteration_big_sleep_time()
    • test_ts_simple_logging()
    • test_ts_simple_usecase()
    • test_ts_wait_for_value_negative()
    • test_ts_wait_for_value_positive()
  • ocs_ci.utility.tests.test_version module
    • test_compare_from_get_semantic_version()
    • test_compare_versions()
    • test_compare_versions_wrong_expression()
    • test_get_semantic_version()
    • test_get_semantic_version_string_values()
  • Module contents
• ocs_ci.utility.workloads package
  • Submodules
  • ocs_ci.utility.workloads.fio module
    • run()
    • setup()
  • ocs_ci.utility.workloads.helpers module
    • find_distro()
  • ocs_ci.utility.workloads.jenkins module
    • run()
    • setup()
  • Module contents

Submodules

ocs_ci.utility.assisted_installer module

This module is for interaction with Assisted Installer API

class ocs_ci.utility.assisted_installer.AccountsMgmtAPI(environment='production')

Bases: OpenShiftAPI

Class for interaction with Accounts mgmt API

get_pull_secret_for_current_user()

Get pull-secret for the current user

Returns:

content of the current user’s pull-secret

Return type:

dict

class ocs_ci.utility.assisted_installer.AssistedInstallerAPI(environment='production')

Bases: OpenShiftAPI

Class for interaction with Assisted Installer API

create_cluster(data)

Create (register) new cluster in Assisted Installer console

Parameters:

data (dict) – cluster configuration based on https://api.openshift.com/?urls.primaryName=assisted-service%20service#/installer/v2RegisterCluster

create_infra_env(data)

Create (register) new Infrastructure Environment in Assisted Installer console

Parameters:

data (dict) – Infrastructure environment configuration based on https://api.openshift.com/?urls.primaryName=assisted-service%20service#/installer/RegisterInfraEnv

delete_cluster(cluster_id)

Delete cluster

Parameters:

cluster_id (str) – cluster ID

delete_infra_env(infra_env_id)

Delete Infrastructure Environment

Parameters:

infra_env_id (str) – Infra environment ID

download_cluster_file(cluster_id, dest_dir, file_name='metadata.json')

Download cluster related files

Parameters:

• cluster_id (str) – cluster ID

• dest_dir (str) – destination directory, where to place the file

• file_name (str) – file to download [metadata.json, bootstrap.ign, master.ign, worker.ign, install-config.yaml, custom_manifests.json, custom_manifests.yaml] (default: metadata.json)

download_cluster_logs(cluster_id, log_dir)

Get cluster logs

Parameters:

• cluster_id (str) – cluster ID

• log_dir (str) – destination directory, where to place the logs

Returns:

the path of the downloaded file

Return type:

str

download_infra_file(infra_env_id, dest_dir, file_name, ipxe_script_type=None, discovery_iso_type=None)

Download Infrastructure Environment related file

Parameters:

• infra_env_id (str) – Infra environment ID

• dest_dir (str) – destination directory, where to place the file

• file_name (str) – file to download [discovery.ign, ipxe-script, static-network-config]

• ipxe_script_type (str) – None or specify the script type to be served for iPXE [‘discovery-image-always’, ‘boot-order-control’]

• discovery_iso_type (str) – None or overrides the ISO type for the disovery ignition [‘full-iso’, ‘minimal-iso’]

Returns:

the path of the downloaded file

Return type:

str

get_cluster(cluster_id)

Get information about one cluster.

Parameters:

cluster_id (str) – cluster ID

Returns:

the cluster configuration

Return type:

dict

get_cluster_admin_credentials(cluster_id)

Get admin credentials and console URL.

Parameters:

cluster_id (str) – cluster ID

Returns:

the credentials and console URL

Return type:

dict

get_cluster_host(cluster_id, host_id)

Get information about host connected to cluster.

Parameters:

• cluster_id (str) – cluster ID

• host_id (str) – host ID

Returns:

the host configuration

Return type:

dict

get_cluster_hosts(cluster_id)

Get information about hosts connected to cluster.

Parameters:

cluster_id (str) – cluster ID

Returns:

the hosts configuration

Return type:

dict

get_cluster_kubeconfig(cluster_id)

Get kubeconfig

Parameters:

cluster_id (str) – cluster ID

Returns:

the kubeconfig content

Return type:

str

get_clusters()

Get list of clusters.

Returns:

the clusters configuration

Return type:

dict

get_component_versions()

Get component versions. Could be used to verify access.

Returns:

the versions

Return type:

dict

get_discovery_iso_url(infra_env_id)

Get Assisted Installer discovery iso url

Parameters:

infra_env_id (str) – Infra environment ID

get_infra_env(infra_env_id)

Get information about one infra_env.

Parameters:

infra_env_id (str) – Infra environment ID

Returns:

the infra_env configuration

Return type:

dict

get_infra_env_host(infra_env_id, host_id)

Get information about host from infra_env.

Parameters:

• infra_env_id (str) – Infra environment ID

• host_id (str) – host ID

Returns:

the host configuration

Return type:

dict

get_infra_env_hosts(infra_env_id)

Get information about hosts from infra_env.

Parameters:

infra_env_id (str) – Infra environment ID

Returns:

the hosts configuration

Return type:

dict

get_infra_envs()

Get list of infra_envs.

Returns:

the infra_envs configuration

Return type:

dict

install_cluster(cluster_id)

Launch installation of the OCP cluster.

Parameters:

cluster_id (str) – cluster ID

Returns:

the cluster configuration

Return type:

dict

update_infra_env_host(infra_env_id, host_id, update_data)

Update host configuration

Parameters:

• infra_env_id (str) – Infra environment ID

• host_id (str) – Host ID

• update_data (dict) – the data to be updated

class ocs_ci.utility.assisted_installer.OpenShiftAPI(environment='production')

Bases: object

Common class for interaction with various OpenShift APIs

property api_base_url

Prepare and return base URL for API

delete_request(endpoint, json=True, ignore_failure=False)

send DELETE request to the API endpoint

Parameters:

• endpoint (str) – the final part of the endpoint URL

• json (bool) – controls if the response should be decoded as json data

• ignore_failure (bool) – controls how to deal with failed request (if raise exception or not)

Returns:

the response from the server (if json == True, parsed as json)

Return type:

Response or dict

get_api_endpoint_url(endpoint='')

prepare and return full api endpoint URL

Parameters:

endpoint (str) – the final part of the endpoint URL

Returns:

full API endpoint URL

Return type:

str

get_request(endpoint, params=None, json=True, ignore_failure=False)

send GET request to the API endpoint

Parameters:

• endpoint (str) – the final part of the endpoint URL

• params (dict) – optional parameters encoded to the request URL

• json (bool) – controls if the response should be decoded as json data

• ignore_failure (bool) – controls how to deal with failed request (if raise exception or not)

Returns:

the response from the server (if json == True, parsed as json)

Return type:

Response or dict

property headers

prepare headers for API requests

patch_request(endpoint, data=None, json=True, ignore_failure=False)

send PATCH request to the API endpoint

Parameters:

• endpoint (str) – the final part of the endpoint URL

• data (dict) – optional data send to the request URL

• json (bool) – controls if the response should be decoded as json data

• ignore_failure (bool) – controls how to deal with failed request (if raise exception or not)

Returns:

the response from the server (if json == True, parsed as json)

Return type:

Response or dict

post_request(endpoint, data=None, json=True, ignore_failure=False)

send POST request to the API endpoint

Parameters:

• endpoint (str) – the final part of the endpoint URL

• data (dict) – optional data send to the request URL

• json (bool) – controls if the response should be decoded as json data

• ignore_failure (bool) – controls how to deal with failed request (if raise exception or not)

Returns:

the response from the server (if json == True, parsed as json)

Return type:

Response or dict

property token

Property for obtaining API token (based on OFFLINE_TOKEN)

ocs_ci.utility.aws module

class ocs_ci.utility.aws.AWS(region_name=None)

Bases: object

This is wrapper class for AWS

append_security_group(security_group_id, instance_id)

Append security group to selected ec2 nodes

Parameters:

• instance_id (str) – Instances to attach security group

• security_group_id (str) – Security group to attach

• out (print) – security group <id> added to selected nodes

attach_role_policy(role_name, policy_arn)

Attach role-policy.

Parameters:

• role_name (str) – Name of the role to attach the policy to

• policy_arn (str) – ARN of the policy to attach to the role

attach_volume(volume, instance_id, device='/dev/sdx')

Attach volume to an ec2 instance

Parameters:

• volume (Volume) – Volume instance

• instance_id (str) – id of instance where to attach the volume

• device (str) – The name of the device where to attach (default: /dev/sdx)

block_instances_access(security_group_id, instances_id)

Block ec2 instances by:

• Append security group without access permissions

• Remove original security groups

Parameters:

• security_group_id (str) – security group without access permissions

• instances_id (list) – list of ec2 instances ids

property cf_client

Property for cloudformation client

Returns:

instance of cloudformation

Return type:

boto3.client

check_volume_attributes(volume_id, name_end=None, size=None, iops=None, throughput=None, namespace=None)

Verify aws volume attributes Primarily used for faas

Parameters:

• volume_id (str) – id of the volume to be checked

• name_end (str) – expected ending of Name tag

• size (int) – expected value of volume’s size

• iops (int) – expected value of IOPS

• throughput (int) – expected value of Throughput

• namespace (str) – expected value of kubernetes.io/created-for/pvc/namespace tag

Raises:

ValueError if the actual value differs from the expected one –

create_hosted_zone(cluster_name)

Create Hosted Zone

Parameters:

cluster_name (str) – Name of cluster

Returns:

Hosted Zone id

Return type:

str

create_iam_role(role_name, description, document)

Create IAM role.

Parameters:

• role_name (str) – Name of the role

• description (str) – Description of the role

• document (str) – JSON string representing the role policy to assume

Returns:

Created role data

Return type:

dict

create_security_group(group_name, dict_permissions, vpc_id)

Create security group with predefined group name and permissions

Parameters:

• group_name (str) – Group name (aws tag: “Group Name”)

• dict_permissions (dict) – The security group’s inbound/outbound permissions

• vpc_id (str) – For group to be attached

Returns:

newly created security group id

Return type:

str

create_stack(s3_url, index, params_list, capabilities)

Create a new cloudformation stack for worker creation

Parameters:

• s3_url (str) – An aws url for accessing s3 object

• index (int) – Integer index for stack name

• params_list (list) – of parameters (k,v) for create_stack

• capabilities (list) – of valid AWS capabilities like CAPABILITY_NAMED_IAM etc

Returns:

of (stack_name, stack_id)

Return type:

tuple

create_volume(availability_zone, name, encrypted=False, size=100, timeout=20, volume_type='gp2')

Create volume

Parameters:

• availability_zone (str) – The availability zone e.g.: us-west-1b

• name (str) – The name of the volume

• encrypted (boolean) – True if encrypted, False otherwise (default: False)

• size (int) – The size in GB (default: 100)

• timeout (int) – The timeout in seconds for volume creation (default: 20)

• volume_type (str) – ‘standard’|’io1’|’gp2’|’sc1’|’st1’ (default: gp2)

Returns:

AWS Resource instance of the newly created volume

Return type:

Volume

create_volume_and_attach(availability_zone, instance_id, name, device='/dev/sdx', encrypted=False, size=100, timeout=20, volume_type='gp2')

Create volume and attach to instance

Parameters:

• availability_zone (str) – The availability zone e.g.: us-west-1b

• instance_id (str) – The id of the instance where to attach the volume

• name (str) – The name of volume

• device (str) – The name of device where to attach (default: /dev/sdx)

• encrypted (boolean) – True if encrypted, False otherwise (default: False)

• size (int) – The size in GB (default: 100)

• timeout (int) – The timeout in seconds for volume creation (default: 20)

• volume_type (str) – ‘standard’|’io1’|’gp2’|’sc1’|’st1’ (default: gp2)

delete_all_record_sets(hosted_zone_id)

Deletes all record sets in a hosted zone

Parameters:

hosted_zone_id (str) – Hosted Zone ID example: /hostedzone/Z91022921MMOZDVPPC8D6

delete_apps_record_set(cluster_name=None, from_base_domain=False)

Delete apps record set that sometimes blocks sg stack deletion.

https://github.com/red-hat-storage/ocs-ci/issues/2549

Parameters:

• cluster_name (str) – Name of the cluster

• from_base_domain (bool) – Delete apps record set from base domain created by Flexy

delete_bucket(bucket)

Delete the bucket

Parameters:

bucket (str) – Name of the bucket to delete

delete_buckets(buckets)

Delete the buckets

Parameters:

buckets (list) – List of buckets to delete

delete_cf_stack_including_dependencies(cfs_name)

Delete cloudformation stack including dependencies.

Some of the depending resources are not deletable, so related errors are ignored and only logged. Thsi method is mainly used as a WORKAROUND for folowing Flexy issue: https://issues.redhat.com/browse/OCPQE-1521

Parameters:

cfs_name (str) – CloudFormation stack name to cleanup

delete_cloudformation_stacks(stack_names)

Delete cloudformation stacks

Parameters:

stack_names (list) – List of cloudformation stacks

delete_hosted_zone(cluster_name, delete_zone=True, delete_from_base_domain=False)

Deletes the hosted zone

Parameters:

• cluster_name (str) – Name of the cluster

• delete_zone (bool) – Whether to delete complete zone

• delete_from_base_domain (bool) – Whether to delete record from base domain

delete_iam_role(role_name)

Delete the specified IAM role.

Parameters:

role_name (str) – Name of the role to delete

delete_objects_in_bucket(bucket)

Delete objects in a bucket

Parameters:

bucket (str) – Name of the bucket to delete objects

delete_record(record, hosted_zone_id)

Deletes the record from Hosted Zone

Parameters:

• record (dict) – record details to delete e.g:{ ‘Name’: ‘vavuthu-eco1.qe.rh-ocs.com.’, ‘Type’: ‘NS’, ‘TTL’: 300, ‘ResourceRecords’:[ {‘Value’: ‘ns-1389.awsdns-45.org’}, {‘Value’: ‘ns-639.awsdns-15.net’}, {‘Value’: ‘ns-1656.awsdns-15.co.uk’}, {‘Value’: ‘ns-183.awsdns-22.com’} ] }

• hosted_zone_id (str) – Hosted Zone ID example: /hostedzone/Z91022921MMOZDVPPC8D6

delete_record_from_base_domain(cluster_name, base_domain=None)

Deletes the record for cluster name in base domain

Parameters:

• cluster_name (str) – Name of the cluster

• base_domain (str) – Base domain name

delete_role_policy(role_name, policy_name)

Delete role policy.

Parameters:

• role_name (str) – Name of the role

• policy_name (str) – Name of the policy to delete

delete_s3_object(bucket_name, object_key)

Delete an object from s3 bucket

Parameters:

• bucket_name (str) – name of a valid s3 bucket

• object_key (str) – the key for s3 object

delete_security_group(security_group_id)

Delete selected security group print out: Security group <id> deleted

Parameters:

security_group_id (str) – Id of selected security group

delete_volume(volume)

Delete an ec2 volume from AWS

Parameters:

volume (Volume) – The volume to delete

detach_and_delete_volume(volume, timeout=120)

Detach volume if attached and then delete it from AWS

Parameters:

• volume (Volume) – The volume to delete

• timeout (int) – Timeout in seconds for API calls

detach_role_policy(role_name, policy_arn)

Detach role policy from IAM role.

Parameters:

• role_name (str) – Name of the role to detach the policy from

• policy_arn (str) – ARN for the policy to detach from the role

detach_volume(volume, timeout=120)

Detach volume if attached

Parameters:

• volume (Volume) – The volume to delete

• timeout (int) – Timeout in seconds for API calls

Returns:

ec2 Volume instance

Return type:

Volume

property ec2_client

Property for ec2 client

Returns:

instance of ec2

Return type:

boto3.client

property ec2_resource

Property for ec2 resource

Returns:

boto3.resource instance of ec2 resource

property elb_client

Property for elb client

Returns:

instance of elb client

Return type:

boto3.client

get_all_security_groups()

Get all security groups in AWS region

Returns:

All security groups

Return type:

list

get_all_stacks()

Get all the cloudformation stacks

Returns:

of all cloudformation stacks

Return type:

list

get_attached_role_policies(role_name)

Get policies attached to a role.

Parameters:

role_name (str) – Name of the role to fetch attached policies

Returns:

Attached role policies

Return type:

list

get_availability_zone_id_by_instance_id(instance_id)

Fetch availability zone out of ec2 node (EC2.Instances.placement)

Parameters:

instance_id (str) – ID of the instance - to get availability zone info from ec2 node

Returns:

availability_zone: The availability zone name

Return type:

str

get_bucket_time_based_rules(bucket_prefixes, bucket_name, hours)

Get the time bucket based prefix and hours

Parameters:

• bucket_prefixes (dict) – The rules according to them determine the number of hours the bucket can exist

• bucket_name (str) – bucket name

• hours (int) – The number of hours bucket can exist if there is no compliance with one of the rules

Returns:

The number of hours bucket can exist

Return type:

int

get_buckets_to_delete(bucket_prefix, hours)

Get the bucket with prefix which are older than given hours

Parameters:

• bucket_prefix (str) – prefix for the buckets to fetch

• hours (int) – fetch buckets that are older than to the specified number of hours

get_caller_identity()

Get STS Caller Identity Account ID

Returns:

Account ID

Return type:

str

get_cloudformation_stacks(pattern)

Get cloudformation stacks

Parameters:

pattern (str) – The pattern of the stack name

get_cluster_subnet_ids(cluster_name)

Get the cluster’s subnet ids of existing cluster

Parameters:

cluster_name (str) – Cluster name

Returns:

string of space separated subnet ids

get_ec2_instance(instance_id)

Get instance of ec2 Instance

Parameters:

instance_id (str) – The ID of the instance to get

Returns:

instance of ec2 instance resource

Return type:

boto3.Instance

get_ec2_instance_volumes(instance_id)

Get all volumes attached to an ec2 instance

Parameters:

instance_id (str) – The ec2 instance ID

Returns:

ec2 Volume instances

Return type:

list

get_hosted_zone_details(zone_id)

Get Hosted zone Details

Parameters:

zone_id (str) – Zone Id of cluster_name

Returns:

Response

Return type:

dict

get_hosted_zone_id(cluster_name)

Get Zone id from given cluster_name

Parameters:

cluster_name (str) – Name of cluster

Returns:

Zone id

Return type:

str

get_hosted_zone_id_for_domain(domain=None)

Get Zone id for domain

Parameters:

domain (str) – Name of the domain.

Returns:

Zone id

Return type:

str

get_iam_roles(base_name)

Retrieve a list of IAM roles that start with the base_name.

Parameters:

base_name (str) – Base of the role name

Returns:

List of IAM roles that match the base_name

Return type:

list

get_instance_id_from_private_dns_name(private_dns_name)

Get the instance id from the private dns name of the instance

Parameters:

private_dns_name (str) – The private DNS name of the instance

Returns:

The instance id associated to the private DNS name.

If not found returns None

Return type:

str

get_instance_profiles_for_role(role_name)

Get instance profiles for the specified role.

Parameters:

role_name (str) – Name of the role to find instance profiles for

Returns:

Instance Profiles for the role

Return type:

list

get_instances_by_name_pattern(pattern)

Get instances by Name tag pattern

The instance details do not contain all the values but just those we are consuming.

Those parameters we are storing for instance are: * id: id of instance * avz: Availability Zone * name: The value of Tag Name if define otherwise None * vpc_id: VPC ID * security_groups: Security groups of the instance

Parameters:

pattern (str) – Pattern of tag name like: pbalogh-testing-cluster-55jx2-worker*

Returns:

contains dictionaries with instance details mentioned above

Return type:

list

get_instances_response_by_name_pattern(pattern=None, filter_by_cluster_name=True)

Get the instances by name tag pattern. If not specified it will return all the instances, or will return the instances filtered by the cluster name.

Parameters:

• pattern (str) – Pattern of tag name like: pbalogh-testing-cluster-55jx2-worker*

• filter_by_cluster_name – Will be used only if the ‘pattern’ param not specified. If True it filters the instances by the cluster name, else if False it returns all instances.

Returns:

list of instances dictionaries.

Return type:

list

get_instances_status_by_id(instance_id)

Get instances by ID

Parameters:

instance_id (str) – ID of the instance

Returns:

The instance status

Return type:

str

get_matching_stacks(pattern)

Get only the stacks which matches the pattern

Parameters:

pattern (str) – A raw string which is re compliant

Returns:

of strings which are matching stack name

Return type:

list

get_ns_for_hosted_zone(zone_id)

Get NameServers Details from Hosted Zone

Parameters:

zone_id (str) – Zone Id of cluster_name

Returns:

NameServers

Return type:

list

get_record_sets(domain=None)

Get all the record sets in domain

Parameters:

domain (str) – Domain name to fetch the records

Returns:

list of record sets

Return type:

list

get_role_policies(role_name)

Get policies embedded in the role.

Parameters:

role_name (str) – Name of the role

Returns:

Role policies

Return type:

list

get_s3_bucket_object_url(bucket_name, object_key)

Get s3 bucket object url

Parameters:

• bucket_name (str) – Name of a valid s3 bucket

• object_key (str) – Name of the key for s3 object

Returns:

An s3 url

Return type:

s3_url (str)

get_security_groups_by_instance_id(instance_id)

Get all attached security groups of ec2 instance

Parameters:

instance_id (str) – Required instance to get security groups from it

Returns:

all_sg_ids: all attached security groups id.

Return type:

list

get_stack_instance_id(stack_name, logical_id)

Get the instance id associated with the cloudformation stack

Parameters:

• stack_name (str) – Name of the cloudformation stack

• logical_id (str) – LogicalResourceId of the resource ex: “Worker0”

Returns:

Id of the instance

Return type:

instance_id (str)

get_stack_name_by_instance_id(instance_id)

Get the stack name by the instance id

Parameters:

instance_id (str) – The instance id

Returns:

The stack name associated to the instance id.

If not found returns None

Return type:

str

get_stack_params(stack_name, param_name)

Get value of a particular param

Parameters:

• stack_name (str) – AWS cloudformation stack name

• param_name (str) – Stack parameter name

Returns:

Parameter value

Return type:

str

get_volume_data(volume_id)

Get volume information

Parameters:

volume_id (str) – ID of the volume

Returns:

complete volume information

Return type:

dict

get_volume_tag_value(volume_data, tag_name)

Get the value of the volume’s tag

Parameters:

• volume_data (dict) – complete volume information

• tag_name (str) – name of the tag

Returns:

value of the tag or None if there’s no such tag

Return type:

str

get_volumes_by_name_pattern(pattern)

Get volumes by pattern

Parameters:

pattern (str) – Pattern of volume name (e.g. ‘cl-vol-‘)

Returns:

Volume information like id and attachments

Return type:

list

get_volumes_by_tag_pattern(tag, pattern)

Get volumes by tag pattern

Parameters:

• tag (str) – Tag name

• pattern (str) – Pattern of tag value (e.g. ‘cl-vol-‘)

Returns:

Volume information like id and attachments

Return type:

list

get_vpc_id_by_instance_id(instance_id)

Fetch vpc id out of ec2 node (EC2.Instances.vpc_id)

Parameters:

instance_id (str) – ID of the instance - to get vpc id info from ec2 node

Returns:

vpc_id: The vpc id

Return type:

str

get_worker_ignition_location(stack_name)

Get the ignition location from given stack

Parameters:

stack_name (str) – AWS cloudformation stack name

Returns:

An AWS URL ignition location

Return type:

ignition_location (str)

get_worker_instance_profile_name(stack_name)

Get the worker instance profile name

Parameters:

stack_name (str) – AWS cloudformation stack name

Returns:

instance profile name

Return type:

worker_instance_profile_name (str)

get_worker_stacks()

Get the cloudformation stacks only for workers of this cluster

Returns:

of worker stacks

Return type:

list

property iam_client

Property for iam client

Returns:

instance of iam

Return type:

boto3.client

list_buckets()

List the buckets

Returns:

List of dictionaries which contains bucket name and creation date as keys

e.g: [ {‘Name’: ‘214qpg-oidc’, ‘CreationDate’: datetime.datetime(2023, 1, 9, 11, 27, 48, tzinfo=tzutc())}, {‘Name’: ‘214rmh4-oidc’, ‘CreationDate’: datetime.datetime(2023, 1, 9, 12, 32, 8, tzinfo=tzutc())} ]

Return type:

list

remove_role_from_instance_profile(role_name, instance_profile_name)

Remove role from instance profile.

Parameters:

• role_name (str) – Name of the role to remove from the instance profile

• instance_profile_name (str) – Name of the instance profile to remove the role from

remove_security_group(security_group_id, instance_id)

Remove security group from selected ec2 instance (by instance id) print out: security group <id> removed from selected nodes

Parameters:

• security_group_id (str) – Security group to be removed

• instance_id (str) – Instance attached with selected security group

restart_ec2_instances(instances)

Restart ec2 instances

Parameters:

instances (dict) – A dictionary of instance IDs and names to restart

restart_ec2_instances_by_stop_and_start(instances, wait=False, force=True)

Restart EC2 instances by stop and start

Parameters:

• instances (dict) – A dictionary of instance IDs and names to stop & start

• wait (bool) – True in case wait for status is needed, False otherwise

• force (bool) – True for force instance stop, False otherwise

restore_instances_access(security_group_id_to_remove, original_security_group_dict)

Restore access to instances by removing blocking security group and append original security group.

Parameters:

• security_group_id_to_remove (str) – id of the security group

• original_security_group_dict (dict) – dict with: keys: blocked instances: ec2 instances id values: list of original security groups

property route53_client

Property for route53 client

Returns:

instance of route53 client

Return type:

boto3.client

property s3_client

Property for s3 client

Returns:

boto3.client instance of s3

property s3_resource

Property for s3 resource

Returns:

boto3.resource instance of s3

start_ec2_instances(instances, wait=False)

Starting an instance

Parameters:

• instances (dict) – A dictionary of instance IDs and names to start

• wait (bool) – True in case wait for status is needed, False otherwise

stop_ec2_instances(instances, wait=False, force=True)

Stopping an instance

Parameters:

• instances (dict) – A dictionary of instance IDs and names to stop

• wait (bool) – True in case wait for status is needed, False otherwise

• force (bool) – True for force instance stop, False otherwise

store_security_groups_for_instances(instances_id)

Stored all security groups attached to selected ec2 instances

Parameters:

instances_id (list) – ec2 instance_id

Returns:

security_group_dict: keys: blocked instances: ec2_instances ids

values: list of original security groups of each instance

Return type:

dict

property sts_client

Property for sts client

Returns:

instance of sts

Return type:

boto3.client

terminate_ec2_instances(instances, wait=True)

Terminate an instance

Parameters:

• instances (dict) – A dictionary of instance IDs and names

• wait (bool) – True in case wait for status is needed, False otherwise

update_hosted_zone_record(zone_id, record_name, data, type, operation_type, ttl=60, raw_data=None)

Update Route53 DNS record

Parameters:

• zone_id (str) – Zone id of DNS record

• record_name (str) – Record Name without domain eg: api.apps.ocp-baremetal-auto

• data (str) – Data to be added for DNS Record

• type (str) – DNS record type

• operation_type (str) – Operation Type (Allowed Values:- Add, Delete)

• ttl (int) – Default set to 60 sec

• raw_data (list) – Data to be added as a record

Returns:

The response from change_resource_record_sets

Return type:

dict

upload_file_to_s3_bucket(bucket_name, object_key, file_path)

Upload objects to s3 bucket

Parameters:

• bucket_name (str) – Name of a valid s3 bucket

• object_key (str) – the key for the s3 object

• file_path (str) – path for the file to be uploaded

wait_for_instances_to_stop(instances)

Wait for the instances to reach status stopped

Parameters:

instances – A dictionary of instance IDs and names

Raises:

botocore.exceptions.WaiterError – If it failed to reach the expected status stopped

wait_for_instances_to_stop_or_terminate(instances)

Wait for the instances to reach statuses stopped or terminated

Parameters:

instances – A dictionary of instance IDs and names

Raises:

botocore.exceptions.WaiterError – If it failed to reach the expected statuses stopped or terminated

wait_for_instances_to_terminate(instances)

Wait for the instances to reach status terminated

Parameters:

instances – A dictionary of instance IDs and names

Raises:

botocore.exceptions.WaiterError – If it failed to reach the expected status terminated

wait_for_record_set(response_list, max_attempts=10)

Wait for Record to be created

Parameters:

• max_attempts (int) – Max Attempt’s for Waiting

• response_list (list) – List of response

exception ocs_ci.utility.aws.AWSTimeoutException

Bases: Exception

exception ocs_ci.utility.aws.StackStatusError

Bases: Exception

ocs_ci.utility.aws.check_root_volume(volume)

Checks whether given EBS volume is root device or not

Parameters:

volume (dict) – EBS volume dictionary

Returns:

True if EBS volume is root device, False otherwise

Return type:

bool

ocs_ci.utility.aws.create_and_attach_ebs_volumes(worker_pattern, size=100, count=1, device_names=('sdx',))

Create volumes on workers

Parameters:

• worker_pattern (string) – Worker name pattern e.g.: cluster-55jx2-worker*

• size (int) – Size in GB (default: 100)

• count (int) – number of EBS volumes to attach to worker node, if it’s

• device_names (list) – list of the devices like [“sda”, “sdb”]. Length of list needs to match count!

Raises:

UnexpectedInput – In case the device_names length doesn’t match count.

ocs_ci.utility.aws.create_and_attach_sts_role()

Create IAM role to support STS deployments.

Returns:

Created role data

Return type:

dict

ocs_ci.utility.aws.create_and_attach_volume_for_all_workers(device_size=None, worker_suffix='worker', count=1, device_letters='ghijklmnopxyz')

Create volumes on workers

Parameters:

• device_size (int) – Size in GB, if not specified value from: config.ENV_DATA[“device_size”] will be used

• worker_suffix (str) – Worker name suffix (default: worker)

• count (int) – number of EBS volumes to attach to worker node

• device_letters (str) – device letters from which generate device names. e.g. for “abc” and if count=2 it will generate [“sda”, “sdb”]

ocs_ci.utility.aws.delete_cluster_buckets(cluster_name)

Delete s3 buckets corresponding to a particular OCS cluster

Parameters:

cluster_name (str) – name of the cluster the buckets belong to

ocs_ci.utility.aws.delete_sts_iam_roles()

Delete IAM roles for the cluster.

ocs_ci.utility.aws.destroy_volumes(cluster_name)

Destroy cluster volumes

Parameters:

cluster_name (str) – The name of the cluster

ocs_ci.utility.aws.get_data_volumes(deviceset_pvs)

Get the instance data volumes (which doesn’t include root FS)

Parameters:

deviceset_pvs (list) – PVC objects of the deviceset PVs

Returns:

ec2 Volume instances

Return type:

list

ocs_ci.utility.aws.get_instances_ids_and_names(instances)

Get the instances IDs and names according to nodes dictionary

Parameters:

instances (list) – Nodes dictionaries, returned by ‘oc get node -o yaml’

Returns:

The ID keys and the name values of the instances

Return type:

dict

ocs_ci.utility.aws.get_rhel_worker_instances(cluster_path)

Get list of rhel worker instance IDs

Parameters:

cluster_path (str) – The cluster path

Returns:

list of instance IDs of rhel workers

Return type:

list

ocs_ci.utility.aws.get_stack_name_from_instance_dict(instance_dict)

Get the stack name by the given instance dictionary from AWS

Parameters:

instance_dict (dict) – The instance dictionary from AWS

Returns:

The stack name of the given instance dictionary from AWS.

If not found returns None

Return type:

str

ocs_ci.utility.aws.get_vpc_id_by_node_obj(aws_obj, instances)

This function getting vpc id by randomly selecting instances out of user aws deployment

Parameters:

• aws_obj (obj) – AWS() object

• instances (dict) – cluster ec2 instances objects

Returns:

vpc_id: The vpc id

Return type:

str

ocs_ci.utility.aws.terminate_rhel_workers(worker_list)

Terminate the RHEL worker EC2 instances

Parameters:

worker_list (list) – Instance IDs of rhel workers

Raises:

exceptions.FailedToDeleteInstance – if failed to terminate

ocs_ci.utility.aws.update_config_from_s3(bucket_name='ocs-ci-data', filename='auth.yaml')

Get the config file that has secrets/configs from the S3 and update the config

Parameters:

• bucket_name (string) – name of the bucket

• filename (string) – name of the file in bucket

Returns:

returns the updated file contents as python dict None: In case the private bucket could not be accessed

Return type:

dict

ocs_ci.utility.azure_utils module

Module for interactions with OCP/OCS Cluster on Azure platform level.

class ocs_ci.utility.azure_utils.AZURE(subscription_id=None, tenant_id=None, client_id=None, client_secret=None, cluster_resource_group=None)

Bases: object

Utility wrapper class for Azure OCP cluster. Design of the class follows similar AWS class.

az_login()

property cluster_resource_group

Azure resource group of the OCP cluster. This group is created by openshift-installer during OCP deployment.

If the value is not yet available and it’s not specified anywhere, it returns None.

property compute_client

Property for Azure vm resource

Returns:

ComputeManagementClient instance for managing Azure vm resource

property credentials

Property for azure service principle credentials used to authenticate the client.

detach_volume(volume, node, timeout=120)

Detach volume if attached

Parameters:

• volume (disk) – disk object required to delete a volume

• node (OCS) – The OCS object representing the node

• timeout (int) – Timeout in seconds for API calls

get_data_volumes(deviceset_pvs)

Get the instance data disk objects

Parameters:

deviceset_pvs (list) – PVC objects of the deviceset PVs

Returns:

Azure Vm disk objects

Return type:

list

get_disk_state(volume_name)

Get the state of the disk

Parameters:

volume_name (str) – Name of the volume/disk

Returns:

Azure Vm disk state

Return type:

str

get_node_by_attached_volume(volume)

Get the Azure Vm instance that has the volume attached to

Parameters:

volume (Disk) – The disk object to get the Azure Vm according to

Returns:

An Azure Vm instance

Return type:

vm

get_storage_account_properties(storage_account_name)

Get the properties of the storage account whose name is passed.

Parameters:

storage_account_name (str) – Name of the storage account

Returns:

Properties of the storage account in string format.

Return type:

str

get_storage_accounts()

Get list of storage accounts in azure resource group

Returns:

list of Azure storage accounts

Return type:

list

get_storage_accounts_names()

Get list of names of storage accounts in azure resource group

Returns:

list of Azure storage accounts name

Return type:

list

get_vm_instance(vm_name)

Get instance of Azure vm Instance

Parameters:

vm_name (str) – The name of the Azure instance to get

Returns:

instance of Azure vm instance resource

Return type:

vm

get_vm_names()

Get list of vms in azure resource group

Returns:

list of Azure vm names

Return type:

(list)

get_vm_power_status(vm_name)

Get the power status of VM

Parameters:

vm_name (str) – Azure VM name

Returns :

str: Power status of Azure VM

property resource_client

Azure ResourceManagementClient instance

restart_vm_instances(vm_names)

Restart Azure vm instances

Parameters:

vm_names (list) – Names of azure vm instances

restart_vm_instances_by_stop_and_start(vm_names, force=False)

Stop and Start Azure vm instances

Parameters:

• vm_names (list) – Names of azure vm instances

• force (bool) – True for non-graceful VM shutdown, False for graceful VM shutdown

start_vm_instances(vm_names)

Start Azure vm instances

Parameters:

vm_names (list) – Names of azure vm instances

stop_vm_instances(vm_names, force=False)

Stop Azure vm instances

Parameters:

• vm_names (list) – Names of azure vm instances

• force (bool) – True for non-graceful VM shutdown, False for graceful VM shutdown

property storage_client

Azure Stroage Management Client instance

class ocs_ci.utility.azure_utils.AzureAroUtil(subscription_id=None, tenant_id=None, client_id=None, client_secret=None, cluster_resource_group=None)

Bases: AZURE

Utility wrapper class for Azure ARO OCP cluster.

create_cluster(cluster_name)

Create OCP cluster.

Parameters:

cluster_name (str) – Cluster name.

Raises:

UnexpectedBehaviour – in the case, the cluster is not installed successfully.

create_network()

Create network related stuff for the cluster.

delete_dns_records(cluster_name, resource_group, base_domain)

Delete DNS records for Azure ARO cluster

Parameters:

• cluster_name (str) – Cluster name.

• resource_group (str) – Base resource group

• base_domain (str) – Base domain for the ARO Cluster

destroy_cluster(cluster_name, resource_group)

Destroy the cluster in Azure ARO.

Parameters:

cluster_name (str) – Cluster name .

get_aro_ocp_version()

Get OCP version available in Azure ARO.

Returns:

version of ARO OCP currently available, matching the version from config.

Return type:

str

Raises:

UnsupportedPlatformVersionError – In case the version is not supported yet available for the Azure ARO.

get_cluster_details(cluster_name)

Returns info about the cluster which is taken from the az command.

Parameters:

cluster_name (str) – Cluster name.

Returns:

Cluster details

Return type:

dict

get_kubeconfig(cluster_name, resource_group, path=None)

Export kubeconfig to provided path.

Parameters:

• cluster_name (str) – Cluster name or ID.

• resource_group (str) – Base resource group

• path (str) – Path where to create kubeconfig file.

set_dns_records(cluster_name, resource_group, base_domain)

Set DNS records for Azure ARO cluster.

Parameters:

• cluster_name (str) – Cluster name.

• resource_group (str) – Base resource group

• base_domain (str) – Base domain for the ARO Cluster

write_kubeadmin_password(cluster_name, resource_group)

Get kubeadmin password for cluster

ocs_ci.utility.azure_utils.azure_storageaccount_check()

Testing that Azure storage account, post deployment.

Testing for property ‘allow_blob_public_access’ to be ‘false’

ocs_ci.utility.azure_utils.load_cluster_resource_group()

Read terraform tfvars.json file created by openshift-installer in a cluster dir to get azure resource group of an OCP cluster. All Azure resources of the cluster are placed in this group.

Returns:

string with resource group name

Raises:

TerrafromFileNotFoundException – When the terraform tfvars file is not found

ocs_ci.utility.azure_utils.load_service_principal_dict(filepath='/home/docs/.azure/osServicePrincipal.json')

Load Azure Service Principal from osServicePrincipal.json file and parse it into a dictionary.

Parameters:

filepath (str) – path of the

Returns:

dictionary with the service principal details (3 IDs and 1 secret)

ocs_ci.utility.baremetal module

class ocs_ci.utility.baremetal.BAREMETAL

Bases: object

wrapper for Baremetal

get_ipmi_ctx(host, user, password)

Function to get ipmi handler :param host: Host mgmt address :type host: str :param user: User Name for accessing mgmt console :type user: str :param password: Password for accessing mgmt console :type password: str

Returns (object): ipmi handler

get_nodes_ipmi_ctx(baremetal_machine)

Get Node Ipmi handler

Parameters:

baremetal_machine – BM objects

get_power_status(ipmi_ctx)

Get BM Power status

Parameters:

ipmi_ctx (object) – Ipmi host handler

Returns: (bool): bm power status

restart_baremetal_machines(baremetal_machine, force=True)

Restart Baremetal Machines

Parameters:

• baremetal_machine (list) – BM objects

• force (bool) – True for BM ungraceful power off, False for graceful BM shutdown

start_baremetal_machines(baremetal_machine, wait=True)

Start Baremetal Machines

Parameters:

• baremetal_machine (list) – BM objects

• wait (bool) – Wait for BMs to start

start_baremetal_machines_with_ipmi_ctx(ipmi_ctxs, wait=True)

Start Baremetal Machines using Ipmi ctx

Parameters:

• ipmi_ctxs (list) – List of BM ipmi_ctx

• wait (bool) – Wait for BMs to start

stop_baremetal_machines(baremetal_machine, force=True)

Stop Baremetal Machines

Parameters:

• baremetal_machine (list) – BM objects

• force (bool) – True for BM ungraceful power off, False for graceful BM shutdown

Raises:

UnexpectedBehaviour – If baremetal machine is still up

verify_machine_is_down(node)

Verifiy Baremetal machine is completely power off

Parameters:

node (object) – Node objects

Returns:

True if machine is down, False otherwise

Return type:

bool

ocs_ci.utility.bootstrap module

ocs_ci.utility.bootstrap.gather_bootstrap()

Gather debugging data for a failing-to-bootstrap control plane. Data is placed in the gather_bootstrap directory under the log directory.

Raises:

NodeNotFoundError – If we are unable to retrieve the IP of any master nodes

ocs_ci.utility.bootstrap.get_gather_bootstrap_node_data()

Retrieve node IPs required by the gather bootstrap command

Raises:

UnsupportedPlatformError – If we do not support gathering bootstrap data for the configured provider

Returns:

Public IP of the bootstrap node and Private IPs of master nodes

Return type:

dict

ocs_ci.utility.bootstrap.get_node_data_aws()

Retrieve bootstrap public IP and master node private IPs running in aws

Raises:

NodeNotFoundError – If we are unable to find the bootstrap node or IP

Returns:

bootstrap and master node IP data

Return type:

dict

ocs_ci.utility.bootstrap.get_node_data_vsphere()

Retrieve bootstrap public IP and master node private IPs running in vsphere

Raises:

NodeNotFoundError – If we are unable to find the bootstrap node or IP

Returns:

bootstrap and master node IP data

Return type:

dict

ocs_ci.utility.cco module

Cloud Credential Operator utility functions

ocs_ci.utility.cco.configure_cloud_credential_operator()

Extract and Prepare the CCO utility (ccoctl) binary. This utility allows us to create and manage cloud credentials from outside of the cluster while in manual mode.

ocs_ci.utility.cco.create_manifests(openshift_installer, output_dir)

Create manifests.

Parameters:

• openshift_installer (str) – Path to the openshift installer

• output_dir (str) – Path to the output directory

ocs_ci.utility.cco.create_service_id(cluster_name, cluster_path, credentials_requests_dir)

Create the Service ID.

Parameters:

• cluster_name (str) – Name of the cluster

• cluster_path (str) – Path to the cluster directory

• credentials_requests_dir (str) – Path to the credentials requests directory

ocs_ci.utility.cco.delete_service_id(cluster_name, credentials_requests_dir)

Delete the Service ID.

Parameters:

• cluster_name (str) – Name of the cluster

• credentials_requests_dir (str) – Path to the credentials requests directory

ocs_ci.utility.cco.extract_ccoctl_binary(cco_image, pull_secret_path)

Extract the ccoctl binary from the CCO container image within the OCP release image.

Parameters:

• cco_image (str) – Release image from the openshift installer

• pull_secret_path (str) – Path to the pull secret

ocs_ci.utility.cco.extract_credentials_requests_aws(release_image, install_config, pull_secret, credentials_requests_dir)

Extract the CredentialsRequests (AWS STS variant).

Parameters:

• release_image (str) – Release image from the openshift installer

• install_config (str) – Location of the install-config.yaml

• credentials_requests_dir (str) – Path to the CredentialsRequests directory

ocs_ci.utility.cco.extract_credentials_requests_ibmcloud(release_image, credentials_requests_dir, pull_secret_path)

Extract the CredentialsRequests (IBM Cloud variant).

Parameters:

• release_image (str) – Release image from the openshift installer

• credentials_requests_dir (str) – Path to the CredentialsRequests directory

• pull_secret_path (str) – Path to the pull secret

ocs_ci.utility.cco.get_cco_container_image(release_image, pull_secret_path)

Obtain the CCO container image from the OCP release image.

Parameters:

• release_image (str) – Release image from the openshift installer

• pull_secret_path (str) – Path to the pull secret

Returns:

ocs_ci.utility.cco.process_credentials_requests_aws(name, aws_region, credentials_requests_dir, output_dir)

Process all CredentialsRequest objects.

Parameters:

• name (str) – Name used to tag any created cloud resources

• aws_region (str) – Region to create cloud resources

• credentials_requests_dir (str) – Path to the CredentialsRequest directory

• output_dir (str) – Path to the output directory

ocs_ci.utility.cco.set_credentials_mode_manual(install_config)

Set credentialsMode to Manual in the install-config.yaml

ocs_ci.utility.connection module

Module that connects to remote server and execute operations on remote server

class ocs_ci.utility.connection.Connection(host, user=None, private_key=None, password=None, stdout=False, jump_host=None)

Bases: object

A class that connects to remote server

exec_cmd(cmd)

Executes command on server

Parameters:

cmd (str) – Command to run on server

Returns:

tuple which contains command return code, output and error

Return type:

tuple

upload_file(localpath, remotepath)

Upload a file to remote server

Parameters:

• localpath (str) – Local file to upload

• remotepath (str) – Target path on the remote server. Filename should be included

ocs_ci.utility.csr module

ocs_ci.utility.csr.approve_csrs(pending_csrs)

Approves the CSRs

Parameters:

csrs (list) – List of CSRs

ocs_ci.utility.csr.approve_pending_csr(expected=None)

After node addition CSR could be in pending state, we have to approve it.

Parameters:

expected (int) – Expected number of CSRs. By default, it will approve all the pending CSRs if exists.

Raises:

• exceptions.PendingCSRException –

• exceptions.TimeoutExpiredError –

ocs_ci.utility.csr.check_no_pending_csr()

Check whether we have any pending CSRs.

Raises:

exceptions.PendingCSRException –

ocs_ci.utility.csr.get_csr_resource()

Retrieve the latest CSR data

Returns:

CSR data

Return type:

ocp.OCP

ocs_ci.utility.csr.get_nodes_csr()

Fetches the CSRs

Returns:

Dictionary with node as keys and CSRs as values

e.g:{

‘control-plane-1’: [‘csr-6hx8z’], ‘compute-2’: [‘csr-blz2n’], ‘compute-0’: [‘csr-p46bz’], ‘control-plane-2’: [‘csr-tzhn5’], ‘control-plane-0’: [‘csr-wm4g5’] }

Return type:

dict

ocs_ci.utility.csr.get_pending_csr()

Gets the pending CSRs

Returns:

list of pending CSRs

Return type:

list

ocs_ci.utility.csr.wait_for_all_nodes_csr_and_approve(timeout=900, sleep=10, expected_node_num=None, ignore_existing_csr=None)

Wait for CSR to generate for nodes

Parameters:

• timeout (int) – Time in seconds to wait

• sleep (int) – Sampling time in seconds

• expected_node_num (int) – Number of nodes to verify CSR is generated

• ignore_existing_csr (dct) – Existing CSR to ignore

• e.g –

  { ‘compute-1’: [‘csr-64vkw’] }

Returns:

True if all nodes are generated CSR

Return type:

bool

Raises:

TimeoutExpiredError – in case CSR not found

ocs_ci.utility.decorators module

ocs_ci.utility.decorators.switch_to_default_cluster_index_at_last(func)

A decorator for switching to the default cluster index after the function execution. This decorator will primarily be used in the ‘teardown’ and ‘finalizer’ methods when we want to make sure that the next test will start with the default cluster index.

Parameters:

func (function) – The function we want to decorate

ocs_ci.utility.decorators.switch_to_orig_index_at_last(func)

A decorator for switching to the original index after the function execution

Parameters:

func (function) – The function we want to decorate

ocs_ci.utility.deployment module

Utility functions that are used as a part of OCP or OCS deployments

ocs_ci.utility.deployment.create_external_secret(ocs_version=None, apply=False)

Creates secret data for external cluster

Parameters:

• ocs_version (str) – OCS version

• apply (bool) – True if want to use apply instead of create command

ocs_ci.utility.deployment.get_and_apply_icsp_from_catalog(image, apply=True, insecure=False)

Get ICSP from catalog image (if exists) and apply it on the cluster (if requested).

Parameters:

• image (str) – catalog image of ocs registry.

• apply (bool) – controls if the ICSP should be applied or not (default: true)

• insecure (bool) – If True, it allows push and pull operations to registries to be made over HTTP

Returns:

path to the icsp.yaml file or empty string, if icsp not available

in the catalog image

Return type:

str

ocs_ci.utility.deployment.get_cluster_prefix(cluster_name, special_rules)

Parse out the “prefix” of a cluster name. Note this is not the same thing as the CLUSTER_PREFIX in jenkins. In fact we will parse that value out. This “cluster prefix” is used to check cloud providers to see if a particular user already has a cluster created. This is to stop people from using too many cloud resources at one time.

Parameters:

• cluster_name (str) – name of the cluster

• special_rules (dict) – dictionary containing special prefix rules that allow clusters to remain alive longer than our default value

Returns:

cluster name prefix

Return type:

str

ocs_ci.utility.deployment.get_ocp_ga_version(channel)

Retrieve the latest GA version for

Parameters:

channel (str) – the OCP version channel to retrieve GA version for

Returns:

latest GA version for the provided channel.

An empty string is returned if no version exists.

Return type:

str

ocs_ci.utility.deployment.get_ocp_release_image()

Get the url of ocp release image * from DEPLOYMENT[“custom_ocp_image”] or * from openshift-install version command output

Returns:

Release image of the openshift installer

Return type:

str

ocs_ci.utility.deployment.get_ocp_release_image_from_installer()

Retrieve release image using the openshift installer.

Returns:

Release image of the openshift installer

Return type:

str

ocs_ci.utility.deployment_openshift_logging module

This module deploys the openshift-logging on the cluster EFK stack

ocs_ci.utility.deployment_openshift_logging.check_health_of_clusterlogging()

• Checks for ElasticSearch, curator, fluentd and kibana pods in

  openshift-logging namespace

• And check for the health of cluster logging, If status is green then the

  cluster is healthy,if status is red then health is bad

Returns:

Gives all the pods that are present in the namespace

Return type:

list

ocs_ci.utility.deployment_openshift_logging.create_clusterlogging_operator_group(yaml_file)

Creation of operator-group for clusterlogging operator.

Parameters:

• yaml_file (str) – Path to yaml file to create operator group for cluster-logging operator

• resource_name (str) – Name of the operator group to create for cluster-logging operator

Returns:

True if operator group for cluster-logging is created

successfully, false otherwise

Return type:

bool

Example

create_clusterlogging_operator_group(yaml_file=constants.CL_OG_YAML)

ocs_ci.utility.deployment_openshift_logging.create_elasticsearch_operator_group(yaml_file, resource_name)

Creation of operator-group for Elastic-search operator

Parameters:

• yaml_file (str) – Path to yaml file to create operator group for elastic-search

• resource_name (str) – Name of the operator group to create for elastic-search

Returns:

True if operator group for elastic search is created

successfully, false otherwise

Return type:

bool

Example:

create_elasticsearch_operator_group(
    constants.EO_OG_YAML, 'openshift-operators-redhat'
)

ocs_ci.utility.deployment_openshift_logging.create_instance()

The function is used to create instance for cluster-logging

ocs_ci.utility.deployment_openshift_logging.create_instance_in_clusterlogging()

Creation of instance for clusterlogging that creates PVCs, ElasticSearch, curator fluentd and kibana pods and checks for all the pods and PVCs

Parameters:

sc_name (str) – Storage class name to create PVCs

Returns:

Contains all detailed information of the

instance such as pods that got created, its resources and limits values, storage class and size details etc.

Return type:

dict

ocs_ci.utility.deployment_openshift_logging.create_namespace(yaml_file)

Creation of namespace “openshift-operators-redhat” for Elasticsearch-operator and “openshift-logging” for ClusterLogging-operator

Parameters:

yaml_file (str) – Path to yaml file to create namespace

Example

create_namespace(yaml_file=constants.EO_NAMESPACE_YAML)

ocs_ci.utility.deployment_openshift_logging.get_clusterlogging_subscription()

Creation of subscription for clusterlogging to subscribe a namespace to an operator

Parameters:

• yaml_file (str) – Path to yaml file to create subscription for the namespace

• resource_name (str) – Name of the subscription

Returns:

Subscription exists or not

Return type:

bool

Example

cl_create_subscription(yaml_file=constants.CL_SUB_YAML)

ocs_ci.utility.deployment_openshift_logging.get_elasticsearch_subscription()

Creation of Subscription for the namespace to subscribe a Namespace to an Operator.

Parameters:

• yaml_file (str) – Path to yaml file to create subscription for a namespace

• resource_name (str) – Name of the subscription

Returns:

Subscription exists or not

Return type:

bool

Example

create_elasticsearch_subscription(constants.EO_SUB_YAML)

ocs_ci.utility.deployment_openshift_logging.install_logging()

ocs_ci.utility.deployment_openshift_logging.set_rbac(yaml_file, resource_name)

Setting Role Based Access Control to grant Prometheus permission to access the openshift-operators-redhat namespace

Parameters:

• yaml_file (str) – Path to yaml file to create RBAC (ROLE BASED ACCESS CONTROL)

• resource_name (str) – Name of the resource for which we give RBAC permissions

Returns:

True if RBAC is set successfully,

false otherwise

Return type:

bool

Example

set_rbac(constants.EO_RBAC_YAML, ‘prometheus-k8s’)

ocs_ci.utility.environment_check module

Util for environment check before and after test to compare and find stale leftovers

ocs_ci.utility.environment_check.assign_get_values(env_status_dict, key, kind=None, exclude_labels=None)

Assigning kind status into env_status_dict

Parameters:

• env_status_dict (dict) – Dictionary which is copy.deepcopy(ENV_STATUS_DICT)

• key (str) – Name of the resource

• kind (OCP obj) – OCP object for a resource

• exclude_labels (list) – App labels to ignore leftovers

ocs_ci.utility.environment_check.compare_dicts(before, after)

Comparing 2 dicts and providing diff list of [added items, removed items]

Parameters:

• before (dict) – Dictionary before execution

• after (dict) – Dictionary after execution

Returns:

List of 2 lists - (‘added’ and ‘removed’ are lists) None: If both parameters are None

Return type:

list

ocs_ci.utility.environment_check.get_environment_status(env_dict, exclude_labels=None)

Get the environment status per kind in KINDS and save it in a dictionary

Parameters:

• env_dict (dict) – Dictionary that is a copy.deepcopy(ENV_STATUS_DICT)

• exclude_labels (list) – App labels to ignore leftovers

ocs_ci.utility.environment_check.get_status_after_execution(exclude_labels=None)

Set the environment status and assign it into ENV_STATUS_PRE dictionary. In addition compare the dict before the execution and after using DeepDiff

Parameters:

exclude_labels (list) – App labels to ignore leftovers

Raises:

ResourceLeftoversException – In case there are leftovers in the environment after the execution

ocs_ci.utility.environment_check.get_status_before_execution(exclude_labels=None)

Set the environment status and assign it into ENV_STATUS_PRE dictionary

Parameters:

exclude_labels (list) – App labels to ignore leftovers

ocs_ci.utility.flexy module

Utility functions related to OCP deployment done via Flexy

ocs_ci.utility.flexy.configure_allowed_domains_in_proxy()

Configure squid proxy server - add domains which needs to be accessible from disconnected cluster for test execution.

ocs_ci.utility.flexy.load_cluster_info(cluster_config=None)

Parse cluster_info.json and cluster_info.yaml files generated by Flexy and load relevant configuration.

Parameters:

cluster_config (MultiClusterConfig) – config object of a specific cluster in case of multiclsuter scenario, if its single cluster we will go ahead with the config

ocs_ci.utility.gcp module

Module for interactions with OCP/OCS Cluster on Google Cloud platform level.

It’s using libcloud module as much as possible, but if that is not feasible, we can use module from Google Cloud python libraries as well. This is not the case so far.

class ocs_ci.utility.gcp.GoogleCloud(project_id=None)

Bases: object

This is a wrapper class for GoogleCloud

base_instance_operation(operation_name, zone, instance_name)

The base method for performing the instance operations: ‘stop’, ‘start’, ‘restart’, ‘terminate’.

Parameters:

• operation_name (str) – The instance operation name(‘stop’, ‘start’, etc.)

• zone – The instance zone

• instance_name – The instance name

Returns:

The result of the execution made by the instance operation

Return type:

dict

base_instances_operation(operation_name, instance_names, wait=True)

The base method for performing the instances operations: ‘stop’, ‘start’, ‘restart’, ‘terminate’.

Parameters:

• operation_name (str) – The operation name to perform on the instances

• instance_names (list) – The instance names

• wait (bool) – If True, will wait for the operation to complete. False, otherwise

Raises:

OperationFailedToCompleteException – In case that not all the operations completed successfully

get_instance_zone_dict()

Get the instance name per instance zone dictionary for all the instances

Returns:

The instance name per instance zone dictionary for all the instances

Return type:

dict

get_instances_zones(instance_names)

Get the zones of the given instance names

Parameters:

instance_names – The instance names to get their zones

Returns:

The zones of the given instance names

Return type:

list

get_operation_data(zone, operation_id)

Get the operation data of a given operation id. (For example after stopping an instance, get the data of the stop operation id)

Parameters:

• zone (str) – The zone of the operation id

• operation_id (str) – The operation id

Returns:

The operation data of a given operation id

Return type:

dict

get_operations_data(zone_operation_id_dict)

Get the operations data for the given operation ids

Parameters:

zone_operation_id_dict – A dictionary of the operation id zone per operation id

Returns:

The operations data for the given operation ids

Return type:

list

restart_instances(instance_names, wait=True)

Restart instances. This is a hard reset - the instance does not do a graceful shutdown

Parameters:

• instance_names (list) – The instance names to restart

• wait (bool) – If True, wait for the instances to be ready. False, otherwise.

Raises:

OperationFailedToCompleteException – If wait is True, and not all the operations completed successfully

restart_instances_by_stop_and_start(instance_names, wait=True)

Restart instances by stop and start

Parameters:

• instance_names (list) – The instance names to restart

• wait (bool) – If True, wait for the instances to be ready. False, otherwise.

Raises:

OperationFailedToCompleteException – If wait is True, and not all the operations completed successfully

start_instances(instance_names, wait=True)

Start instances

Parameters:

• instance_names (list) – The instance names to start

• wait (bool) – If True, wait for the instances to be ready. False, otherwise.

Raises:

OperationFailedToCompleteException – If wait is True, and not all the operations completed successfully

stop_instances(instance_names, wait=True)

Stop instances

Parameters:

• instance_names (list) – The instance names to stop

• wait (bool) – If True, wait for the instances to stop. False, otherwise.

Raises:

OperationFailedToCompleteException – If wait is True, and not all the operations completed successfully

terminate_instances(instance_names, wait=True)

Terminate instances

Parameters:

• instance_names (list) – The instance names to terminate

• wait (bool) – If True, wait for the instances to terminate. False, otherwise.

Raises:

OperationFailedToCompleteException – If wait is True, and not all the operations completed successfully

wait_for_operations_to_complete(zone_operation_id_dict, timeout=300, sleep=10)

Wait for the operations with the given operation IDs to complete

Parameters:

• zone_operation_id_dict (dict) – The operation zone per operation id dictionary

• timeout (int) – Time in seconds to wait for the operation to complete

• sleep (int) – Time in seconds to wait between iterations

Raises:

OperationFailedToCompleteException – In case that not all the operations completed successfully

class ocs_ci.utility.gcp.GoogleCloudUtil(region_name=None)

Bases: object

Utility wrapper class for Google Cloud OCP cluster. Design of the class follows similar AWS and Azure class.

property compute_driver

Compute Driver instance for GCP.

property service_account

Dictionary with GCP service account details, which contains authentication keys and cluster details loaded from Service Account Key file.

ocs_ci.utility.gcp.SERVICE_ACCOUNT_KEY_FILEPATH = '/home/docs/.gcp/osServiceAccount.json'

absolute filepath of json file with service account key

This is json key file of sg-serv-account service account, which has full admin rights in given GCP project. The same key file is used by openshift installer during OCP installation to create all cluster resources from virtual machines to hostnames. Modules from ocs-ci are using the same key to get full cluster access as well.

For more details, see GCP documentation on ServiceAccountKey resource

Type:

str

ocs_ci.utility.gcp.load_service_account_key_dict(filepath='/home/docs/.gcp/osServiceAccount.json')

Load GCP Service Account key from osServiceAccount.json file and parse it into a dictionary.

Parameters:

filepath (str) – path of the osServiceAccount.json file

Returns:

dictionary with the service account details

ocs_ci.utility.ibmcloud module

Module for interactions with IBM Cloud Cluster.

class ocs_ci.utility.ibmcloud.IBMCloud

Bases: object

Wrapper for Ibm Cloud

attach_volume(volume, node)

Attach volume to node on IBM Cloud.

Parameters:

• volume (str) – volume id.

• node (OCS) – worker node id to attach.

delete_volume_id(volume)

Deletes Volumeid

Parameters:

volume (str) – The volume to be deleted

detach_volume(volume, node=None, delete_from_backend=True)

Detach volume from node on IBM Cloud.

Parameters:

• volume (str) – volume id.

• node (OCS) – worker node id to detach.

• delete_from_backend (bool) – True for deleting the disk from the storage backend, False otherwise

get_data_volumes()

Returns volumes in IBM Cloud for cluster.

Returns:

volumes in IBM Cloud for cluster.

Return type:

list

get_node_by_attached_volume(volume)

Get the node by attached volume on IBM Cloud.

Parameters:

volume (str) – volume id.

Raises:

NodeHasNoAttachedVolume – In case the volume is not attached to node

Returns:

worker id

Return type:

str

get_volume_id()

Returns Volumeid with the name taken from constants

Returns:

volume id if the volume exists otherwise create

new volume

Return type:

str

is_volume_attached(volume)

Check if volume is attached to node or not.

Parameters:

volume (str) – The volume to check for to attached

Returns:

‘True’ if volume is attached otherwise ‘False’

Return type:

bool

restart_nodes(nodes, timeout=900, wait=True)

Reboot the nodes on IBM Cloud.

Parameters:

nodes (list) – The worker node instance

restart_nodes_by_stop_and_start(nodes, force=True)

Reboot the nodes which are not ready on IBM Cloud.

Parameters:

• nodes (list) – The OCS objects of the nodes

• force (bool) – True for force VM stop, False otherwise

wait_for_volume_attach(volume)

Checks volume is attached to node or not

Parameters:

volume (str) – The volume to wait for to be attached

Returns:

True if the volume has been attached to the

instance, False otherwise

Return type:

bool

ocs_ci.utility.ibmcloud.add_deployment_dependencies()

Adding dependencies for IBM Cloud deployment

ocs_ci.utility.ibmcloud.create_cluster(cluster_name)

Create OCP cluster.

Parameters:

cluster_name (str) – Cluster name.

Raises:

UnexpectedBehaviour – in the case, the cluster is not installed successfully.

ocs_ci.utility.ibmcloud.destroy_cluster(cluster)

Destroy the cluster on IBM Cloud.

Parameters:

cluster (str) – Cluster name or ID.

ocs_ci.utility.ibmcloud.get_cluster_details(cluster)

Returns info about the cluster which is taken from the ibmcloud command.

Parameters:

cluster (str) – Cluster name or ID

ocs_ci.utility.ibmcloud.get_ibmcloud_ocp_version()

Get OCP version available in IBM Cloud.

ocs_ci.utility.ibmcloud.get_kubeconfig(cluster, path)

Export kubeconfig to provided path.

Parameters:

• cluster (str) – Cluster name or ID.

• path (str) – Path where to create kubeconfig file.

ocs_ci.utility.ibmcloud.get_region(cluster_path)

Get region from metadata.json in given cluster_path

Parameters:

cluster_path – path to cluster install directory

Returns:

region where cluster is deployed

Return type:

str

ocs_ci.utility.ibmcloud.is_cluster_installed(cluster)

Check if cluster is installed and return True if so, False otherwise.

Parameters:

cluster (str) – Cluster name or ID

ocs_ci.utility.ibmcloud.label_nodes_region()

Apply the region label to the worker nodes. Necessary for IBM COS-backed backingstore support.

ocs_ci.utility.ibmcloud.list_clusters(provider=None)

Returns info about the cluster which is taken from the ibmcloud command.

Parameters:

provider (str) – Provider type (classic, vpc-classic, vpc-gen2).

ocs_ci.utility.ibmcloud.login()

Login to IBM Cloud cluster

ocs_ci.utility.ibmcloud.run_ibmcloud_cmd(cmd, secrets=None, timeout=600, ignore_error=False, **kwargs)

Wrapper function for run_cmd which if needed will perform IBM Cloud login command before running the ibmcloud command. In the case run_cmd will fail because the IBM cloud got disconnected, it will login and re-try.

Parameters:

• cmd (str) – command to run

• secrets (list) – A list of secrets to be masked with asterisks This kwarg is popped in order to not interfere with subprocess.run(**kwargs)

• timeout (int) – Timeout for the command, defaults to 600 seconds.

• ignore_error (bool) – True if ignore non zero return code and do not raise the exception.

ocs_ci.utility.ibmcloud.set_region()

Sets the cluster region to ENV_DATA

ocs_ci.utility.ibmcloud_bm module

Module for interactions with IBM Cloud Cluster.

class ocs_ci.utility.ibmcloud_bm.IBMCloudBM(region=None)

Bases: object

Wrapper for IBM Cloud with Bare metal machines

get_all_machines()

Get all the machines in the IBMCloud Bare metal machines

Returns:

List of dictionaries. List of all the machines in the IBMCloud Bare metal machines

Return type:

list

get_machines_by_names(machine_names)

Get the machines in the IBMCloud Bare metal machines that have the given machine names

Parameters:

machine_names (list) – The list of the machine names to search for.

Returns:

Get the machines in the IBMCloud Bare metal machines that have the given machine names

login()

Login to IBM Cloud account

restart_machines(machines, force=False)

Reboot the IBMCloud Bare metal machines

Parameters:

• machines (list) – List of the IBMCLoud Bare metal machines objects to restart

• force (bool) – If False, will perform a soft reboot. Otherwise, if True, will perform a hard reboot

restart_machines_by_stop_and_start(machines)

Restart the IBMCloud Bare metal machines by stop and start

Parameters:

machines (list) – List of the IBMCLoud Bare metal machines objects to restart

run_ibmcloud_bm_cmd(cmd, secrets=None, timeout=600, ignore_error=False, **kwargs)

Wrapper function for run_cmd which if needed will perform IBM Cloud login command before running the ibmcloud bare metal command. In the case run_cmd will fail because the IBM cloud got disconnected, it will login and re-try.

Parameters:

• cmd (str) – command to run

• secrets (list) – A list of secrets to be masked with asterisks This kwarg is popped in order to not interfere with subprocess.run(**kwargs)

• timeout (int) – Timeout for the command, defaults to 600 seconds.

• ignore_error (bool) – True if ignore non zero return code and do not raise the exception.

start_machines(machines)

Start the IBMCloud Bare metal machines

Parameters:

machines (list) – List of the IBMCLoud Bare metal machines objects to start

stop_machines(machines)

Stop the IBMCloud Bare metal machines

Parameters:

machines (list) – List of the IBMCLoud Bare metal machines objects to stop

ocs_ci.utility.ipam module

This module will interact with IPAM server

class ocs_ci.utility.ipam.IPAM(appiapp, ipam=None, token=None)

Bases: object

IPAM class

assign_ip(host, subnet)

Reserve IP in IPAM server for a given host

Parameters:

• host (str) – hostname to reserve IP in IPAM server

• subnet (str) – subnet to reserve IP

Returns:

Reserved IP

Return type:

str

Raises:

• HTTPError – in case of HTTP error

• ConnectionError – in case of Connection Error

• Timeout – in case of Timeout

• RequestException – Any Exception from requests

• IPAMAssignUpdateFailed – if it fails to assign IP

assign_ips(hosts, subnet)

Assign IPs to the hosts

Parameters:

• hosts (list) – List of hosts to reserve IP in IPAM server

• subnet (str) – subnet to reserve IPs

Returns:

List of Reserved IP’s

Return type:

list

release_ip(hostname)

Release IP from IPAM server

Parameters:

hostname (str) – Hostname to release IP

Raises:

IPAMReleaseUpdateFailed – If it fails to release IP from IPAM server

release_ips(hosts)

Releases host IP’s from IPAM server

Parameters:

hosts (list) – List of host names to release IP’s

ocs_ci.utility.kms module

This module contains KMS related class and methods currently supported KMSs: Vault and HPCS

class ocs_ci.utility.kms.HPCS

Bases: KMS

A class which handles deployment and other configs related to HPCS

cleanup()

Cleanup the backend resources in case of external

create_hpcs_csi_kms_connection_details(namespace='openshift-storage')

Create hpcs specific csi kms connection details configmap resource

create_ibm_kp_kms_secret(namespace='openshift-storage')

create hpcs specific csi kms secret resource

create_ocs_hpcs_resources()

This function takes care of creating ocp resources for secrets like hpcs customer root key, service api key, etc. Assumption is hpcs section in AUTH file contains hpcs service instance id, base url, token url, api key and customer root key.

delete_resource(resource_name, resource_type, resource_namespace)

Given resource type, resource name and namespace, this function will delete oc resource

Parameters:

• resource_name – name of the resource

• resource_type – type of resource such as secret

• resource_namespace – namespace in which resource is present

deploy()

This function delegates the deployment of hpcs based on OCP or vault standalone external mode deployment

deploy_hpcs_external()

This function takes care of deployment and configuration for external mode hpcs deployment. We are assuming that an external hpcs service already exists and we will be just configuring the necessary OCP objects for OCS like secrets, token etc

gather_hpcs_config()

This function populates the hpcs configuration

gather_init_hpcs_conf()

Gather hpcs configuration and init the vars This function currently gathers only for external mode

get_token_for_ibm_api_key()

This function retrieves the access token in exchange of an IBM API key

Returns:

access token for authentication with IBM endpoints

Return type:

(str)

list_hpcs_keys()

This function lists the keys present in a HPCS instance

Returns:

list of keys in a HPCS instance

Return type:

(list)

post_deploy_verification()

Validating the OCS deployment from hpcs perspective

validate_external_hpcs()

This function is for post OCS deployment HPCS verification

Following checks will be done 1. check osd encryption keys in the HPCS path 2. check noobaa keys in the HPCS path 3. check storagecluster CR for ‘kms’ enabled

Raises:

NotFoundError – if key not found in HPCS OR in the resource CR

class ocs_ci.utility.kms.KMIP

Bases: KMS

A class which handles deployment and other configs related to KMIP (Thales CipherTrust Manager)

check_key_exists_in_ciphertrust(key_id)

Check if a key with the key_id is present in CipherTrust Manager

Parameters:

key_id (str) – ID of the key to be checked for

Returns:

True, if the key exists in CipherTrust Manager

Return type:

(bool)

cleanup()

Cleanup for KMIP

create_ciphertrust_key(key_name)

Create a key in Ciphertrust Manager to be used for PV encryption

Parameters:

key_name (str) – Name of the key to be created

Returns:

ID of the key created in CipherTrust

Return type:

(str)

create_kmip_csi_kms_connection_details(namespace='openshift-storage')

Create KMIP specific csi-kms-connection-details configmap resource

create_kmip_secret(type='ocs')

Create secret containing the certificates and unique identifier (only for PV encryption) from CipherTrust Manager KMS

Parameters:

type (str) – csi, if the secret is being created for PV/Storageclass encryption ocs, if the secret is for clusterwide encryption

Returns:

name of the kmip secret

Return type:

(str)

create_odf_kmip_resources()

Create secret containing certs and the ocs-kms-connection-details confignmap

delete_ciphertrust_key(key_id)

Delete key from CipherTrust Manager

Parameters:

key_id (str) – ID of the key to be deleted

deploy()

This function delegates the deployment of KMS using KMIP. Thales CipherTrust Manager is the only supported vendor for now.

deploy_kmip_ciphertrust()

This function configures the resources required to use Thales CipherTrust Manager with ODF

get_key_info_ciphertrust(key_id)

Retrieve information about a given key

Parameters:

key_id (str) – ID of the key in CipherTrust

Returns

(dict): Dictionary with key details

get_key_list_ciphertrust()

Lists all keys in CipherTrust Manager

Returns:

list containing the IDs of the keys

Return type:

(list)

get_noobaa_key_id()

Retrieve the key ID used for encryption by Noobaa

Returns:

Key ID used by NooBaa for encryption

Return type:

(str)

get_osd_key_ids()

Retrieve the key ID used for OSD encryption stored in their respective secrets

Returns:

list of key IDs from all OSDs

Return type:

(list)

post_deploy_verification()

Verify ODF deployment using KMIP

update_kmip_env_vars()

Set the environment variable for CipherTrust to allow running ksctl CLI cmds

validate_ciphertrust_deployment()

Verify whether OSD and NooBaa keys are stored in CipherTrust Manager

class ocs_ci.utility.kms.KMS(provider=None)

Bases: object

This is base class for any KMS integration

create_csi_kms_resources()

create_resource(resource_data, prefix=None)

Given a dictionary of resource data, this function will creates oc resource

Parameters:

• resource_data (dict) – yaml dictionary for resource

• prefix (str) – prefix for NamedTemporaryFile

deploy()

post_deploy_verification()

class ocs_ci.utility.kms.Vault

Bases: KMS

A class which handles deployment and other configs related to vault

cleanup()

Cleanup the backend resources in case of external

create_namespace(vault_namespace)

Create a vault namespace

Parameters:

vault_namespace (str) – name of the vault namespace

Raises:

VaultOperationError – If namespace is not created successfully

create_ocs_kube_auth_resources(sa_name='odf-vault-auth')

This function will create the serviceaccount and clusterrolebindings required for kubernetes auth

Parameters:

sa_name (str) – Name of the service account in ODF

create_ocs_vault_cert_resources()

Explicitly create secrets like ca cert, client cert, client key Assumption is vault section in AUTH file contains base64 encoded (with padding) ca, client certs, client key

create_ocs_vault_resources()

This function takes care of creating ocp resources for secrets like ca cert, client cert, client key and vault token Assumption is vault section in AUTH file contains base64 encoded (with padding) ca, client certs, client key and vault path token

create_tenant_configmap(tenant_namespace, **vault_config)

This functional will create a configmap in the tenant namespace to override the vault config in csi-kms-connection-details configmap.

Parameters:

• tenant_namespace (str) – Tenant namespace

• vaultBackend (str) – KV version to be used, either kv or kv-v2

• vaultBackendPath (str) – The backend path in Vault where the encryption keys will be stored

• vaultNamespace (str) – Namespace in Vault, if exists, where the backend path is created

• vaultRole (str) – (Vaulttenantsa) The role name in Vault configured with kube auth method for the given policy and tenant namespace

• vaultAuthPath (str) – (Vaulttenantsa) The path where kubernetes auth method is enabled

• vaultAuthNamespace (str) – (Vaulttenantsa) The namespace where kubernetes auth method is enabled, if exists

create_tenant_sa(namespace)

This function will create the serviceaccount in the tenant namespace to authenticate to Vault when vaulttenantsa KMS type is used for PV encryption.

Parameters:

namespace (str) – The tenant namespace where the service account will be created

create_token_reviewer_resources()

This function will create the rbd-csi-vault-token-review SA, clusterRole and clusterRoleBindings required for the kubernetes auth method with vaulttenantsa encryption type.

Raises:

CommandFailed – Exception if the command fails

create_vault_csi_kms_connection_details(kv_version, vault_auth_method='vaulttokens', namespace=None)

Create vault specific csi kms connection details configmap resource

create_vault_csi_kms_token(namespace=None)

create vault specific csi kms secret resource

Parameters:

namespace (str) – then value from config will be used.

create_vault_kube_auth_role(namespace, role_name='csi-kubernetes', sa_name='ceph-csi-vault-sa')

Create a role for tenant authentication in Vault

Parameters:

• namespace (str) – namespace in ODF cluster

• role_name (str) – Name of the role in Vault

• sa_name (str) – Service account in the tenant namespace to be used for authentication

deploy()

This function delegates the deployment of vault based on OCP or vault standalone external mode deployment

deploy_vault_external()

This function takes care of deployment and configuration for external mode vault deployment. We are assuming that an external vault service already exists and we will be just configuring the necessary OCP objects for OCS like secrets, token etc

deploy_vault_internal()

This function takes care of deployment and configuration for internal mode vault deployment on OCP

gather_init_vault_conf()

Gather vault configuration and init the vars This function currently gathers only for external mode Gathering for internal mode woulde be different

gather_vault_config()

This function populates the vault configuration

generate_vault_token()

Generate a token for self.vault_policy_name

Returns:

vault token

Return type:

str

get_vault_backend_path()

Fetch the vault backend path used for this deployment This can be obtained from kubernetes secret resource ‘ocs-kms-connection-details’

apiVersion: v1
data:
  KMS_PROVIDER: vault
  KMS_SERVICE_NAME: vault
  VAULT_ADDR: https://xx.xx.xx.xx:8200
  VAULT_BACKEND_PATH: ocs

get_vault_connection_info(resource_name=None)

Get resource info from ocs-kms-connection-defatils

Parameters:

resource_name (str) – name of the resource

get_vault_kube_auth_role()

Fetch the role name from ocs-kms-connection-details configmap

get_vault_namespace()

From kms connection details resource obtain namespace

get_vault_path_token()

Fetch token from kubernetes secret we need this to find the vault policy default name in case of ocs is ‘ocs-kms-token’

apiVersion: v1
data:
  token: cy5DRXBKV0lVbzNFQjM1VHlGMFNURzZQWms=
kind: Secret
metadata:
  name: ocs-kms-token
namespace: openshift-storage
type: Opaque

get_vault_policy()

Get the policy name based on token from vault

post_deploy_verification()

Validating the OCS deployment from vault perspective

remove_vault_backend_path(vault_namespace=None)

remove vault path

Parameters:

vault_namespace (str) – Namespace in Vault, if exists, where the backend path is created

remove_vault_namespace()

Cleanup the namespace

Raises:

KMSResourceCleanupError – If namespace deletion fails

remove_vault_policy(vault_namespace=None)

Cleanup the policy we used

Parameters:

namespace (vault) – Namespace in Vault, if exists, where the backend path is created

setup_vault_client_cert()

For Vault cli interaction with the server we need client cert to talk to HTTPS on the vault server

update_vault_env_vars()

In order to run vault CLI we need following env vars VAULT_ADDR and VAULT_TOKEN

validate_external_vault()

This function is for post OCS deployment vault verification

Following checks will be done 1. check osd encryption keys in the vault path 2. check noobaa keys in the vault path 3. check storagecluster CR for ‘kms’ enabled

Raises:

NotFoundError – if key not found in vault OR in the resource CR

vault_backend_path_exists(backend_path)

Check if vault backend path already exists

Parameters:

backend_path (str) – name of the vault backend path

Returns:

True if exists else False

Return type:

bool

vault_create_backend_path(backend_path=None, kv_version=None)

create vault path to be used by OCS

Raises:

VaultOperationError exception –

vault_create_namespace(namespace=None)

Create a vault namespace if it doesn’t exists

vault_create_policy(policy_name=None)

Create a vault policy and generate token

Raises:

VaultOperationError exception –

vault_kube_auth_setup(auth_path=None, auth_namespace=None, token_reviewer_name='rbd-csi-vault-token-review')

Setup kubernetes auth method in Vault

Parameters:

• auth_path (str) – The path where kubernetes auth is to be enabled. If not provided default ‘kubernetes’ path is used

• auth_namespace (str) – The vault namespace where kubernetes auth is to be enabled, if applicable

• token_reviewer_name (str) – Name of the token-reviewer serviceaccount in openshift-storage namespace

Raises:

VaultOperationError – if kube auth method setup fails

vault_namespace_exists(vault_namespace)

Check if vault namespace already exists

Parameters:

vault_namespace (str) – name of the vault namespace

Returns:

True if exists else False

Return type:

bool

vault_sealed()

Returns:

if vault is sealed then return True else False

Return type:

bool

vault_unseal()

Unseal vault if sealed

Raises:

VaultOperationError – In case unseal operation failed

ocs_ci.utility.kms.get_encryption_kmsid()

Get encryption kmsid from ‘csi-kms-connection-details’ configmap resource

Returns:

A list of KMS IDs available

Return type:

kmsid (list)

Raises:

KMSConnectionDetailsError – if csi kms connection detail doesn’t exist

ocs_ci.utility.kms.get_kms_deployment()

ocs_ci.utility.kms.get_kms_endpoint()

Fetch VAULT_ADDR or KMIP_ENDPOINT if kmip provider from ocs-kms-connection-details configmap.

Returns:

KMS endpoint address

Return type:

str

ocs_ci.utility.kms.get_ksctl_cli(bin_dir=None)

Download ksctl to interact with CipherTrust Manager via CLI

Parameters:

bin_dir (str) – Path to bin directory (default: config.RUN[‘bin_dir’])

ocs_ci.utility.kms.is_key_present_in_path(key, path)

Check if key is present in the backend Path

Parameters:

• key (str) – Name of the key

• path (str) – Vault backend path name

Returns:

True if key is present in the backend path

Return type:

(bool)

ocs_ci.utility.kms.is_kms_enabled(dont_raise=False)

Checks StorageCluster yaml if kms is configured.

Returns:

True if KMS is configured else False

Return type:

(bool)

ocs_ci.utility.kms.remove_kmsid(kmsid)

This function will remove all the details for the given kmsid from the csi-kms-connection-details configmap

Parameters:

kmsid (str) – kmsid to be remove_kmsid

Raises:

KMSResourceCleaneupError – If the kmsid entry is not deleted

ocs_ci.utility.kms.remove_token_reviewer_resources()

Delete the SA, clusterRole and clusterRoleBindings for token reviewer

ocs_ci.utility.kms.set_kms_endpoint(address)

Set VAULT_ADDR or KMIP_ENDPOINT if kmip provider in ocs-kms-connection-details configmap to provided value

Parameters:

address (str) – Address to be set in KMS configuration

ocs_ci.utility.kms.update_csi_kms_vault_connection_details(update_config)

Update the vault connection details in the resource csi-kms-connection-details

Parameters:

update_config (dict) – A dictionary of vault info to be updated

ocs_ci.utility.kms.vault_kv_list(path)

List kv from a given path

Parameters:

path (str) – Vault backend path name

Returns:

of kv present in the path

Return type:

list

ocs_ci.utility.load_balancer module

Module that contains all operations related to load balancer in a cluster

class ocs_ci.utility.load_balancer.LoadBalancer(host=None, user=None, private_key=None)

Bases: object

A class that handles all operations related to load balancer

modify_haproxy_service()

Modify haproxy service

reload_daemon()

Reload daemon-reload

remove_boostrap_in_proxy()

Removes bootstrap IP from haproxy.conf

remove_compute_node_in_proxy()

Removes compute node IP’s from haproxy.conf

rename_haproxy()

Rename haproxy configuration file from haproxy.conf to haproxy.cfg

rename_haproxy_conf_and_reload()

Rename haproxy config and restart haproxy service

restart_haproxy()

Restarts haproxy service

Returns:

True if successful restarts of haproxy, False otherwise

Return type:

bool

restart_service(service_name)

Restarts the given service

update_haproxy_with_nodes(nodes)

Parameters:

nodes (list) – List of nodes to update in haproxy

ocs_ci.utility.localstorage module

This module contains local-storage related methods

ocs_ci.utility.localstorage.check_local_volume_local_volume_set()

Function to check if Local-volume and Local volume set is present or not

Returns:

dict for localvolume and localvolumeset

Return type:

dict

ocs_ci.utility.localstorage.check_pvs_created(num_pvs_required)

Verify that exact number of PVs were created and are in the Available state

Parameters:

num_pvs_required (int) – number of PVs required

Raises:

AssertionError – if the number of PVs are not in the Available state

ocs_ci.utility.localstorage.fetch_all_device_paths()

Return all device paths inside worker nodes

Returns:

List containing all device paths

Return type:

list

ocs_ci.utility.localstorage.get_local_volume_cr()

Get localVolumeCR object

Returns:

Local Volume object handler

Return type:

local volume (obj)

ocs_ci.utility.localstorage.get_lso_channel()

Get the channel to use for installing the local storage operator

Returns:

local storage operator channel

Return type:

str

ocs_ci.utility.localstorage.get_new_device_paths(device_sets_required, osd_size_capacity_requested)

Get new device paths to add capacity over Baremetal cluster

Parameters:

• device_sets_required (int) – Count of device sets to be added

• osd_size_capacity_requested (int) – Requested OSD size capacity

Returns:

List containing added device paths

Return type:

list

ocs_ci.utility.lvm_utils module

ocs_ci.utility.lvm_utils.lvcreate(osd, lv_name, vg_name, size)

ocs_ci.utility.lvm_utils.make_partition(osd, device, start=None, end=None, gpt=False)

ocs_ci.utility.lvm_utils.osd_scenario1(osd, devices_dict, dmcrypt=False)

OSD scenario type1 generator :param osd: osd node :param devices_dict: dict of devices of the osd node supplied :param dmcrypt: False by default

Returns:

generated scenario, dmcrypt

ocs_ci.utility.lvm_utils.osd_scenario1_dmcrypt(osd, devices_dict)

OSD scenario type2 generator :param osd: osd node :param devices_dict: dict of devices of the osd node supplied :param dmcrypt: False by default

Returns:

generated scenario, dmcrypt(overridden to True)

ocs_ci.utility.lvm_utils.osd_scenario2(osd, devices_dict, dmcrypt=False)

OSD scenario type3 generator :param osd: osd node :param devices_dict: dict of devices of the osd node supplied :param dmcrypt: False by default

Returns:

generated scenario, dmcrypt

ocs_ci.utility.lvm_utils.osd_scenario2_dmcrypt(osd, devices_dict)

OSD scenario type4 generator :param osd: osd node :param devices_dict: dict of devices of the osd node supplied :param dmcrypt: False by default

Returns:

generated scenario, dmcrypt(overridden to True)

ocs_ci.utility.lvm_utils.osd_scenario3(osd, devices_dict, dmcrypt=False)

OSD scenario type5 generator :param osd: osd node :param devices_dict: dict of devices of the osd node supplied :param dmcrypt: False by default

Returns:

generated scenario, dmcrypt

ocs_ci.utility.lvm_utils.osd_scenario3_dmcrypt(osd, devices_dict)

OSD scenario type6 generator :param osd: osd node :param devices_dict: dict of devices of the osd node supplied :param dmcrypt: False by default

Returns:

generated scenario, dmcrypt(overridden to True)

ocs_ci.utility.lvm_utils.pvcreate(osd, devices)

ocs_ci.utility.lvm_utils.vgcreate(osd, vg_name, devices)

ocs_ci.utility.lvmo_utils module

ocs_ci.utility.lvmo_utils.delete_lvm_cluster()

Delete lvm cluster if exists

Raises:

execption if lvmcluster cant be deleted –

ocs_ci.utility.lvmo_utils.get_lvm_cluster_name()

Get LVM clustername Dynamically

Returns:

(str) lvm cluster name.

ocs_ci.utility.lvmo_utils.get_sno_blockdevices(node='sno-edge-0')

Gets list of storage devices by it’s names

Parameters:

node (str) – node name

Returns:

list of storage devices full-names (str)

Return type:

list

ocs_ci.utility.lvmo_utils.get_sno_disks_by_path(node='sno-edge-0')

Get list of storage devices by it’s path as listed on node

Parameters:

node (str) – node name

Returns:

list of storage devices full-path (str)

Return type:

list

ocs_ci.utility.lvmo_utils.lvmo_health_check(tries=20, delay=30)

Priodic check LVMO cluster health

Parameters:

• tries (int) – number of retries.

• delay (int) – delay between retries.

Returns:

True if cluster is healthy, raise exception otherwise.

Return type:

bool

ocs_ci.utility.lvmo_utils.lvmo_health_check_base()

Check health of lvm cluster by checking the following: lvm cluster cr status, lvm pods status

Returns:

True if all checks passed, raise exception otherwise

Return type:

bool

ocs_ci.utility.managedservice module

ocs_ci.utility.managedservice.generate_onboarding_token()

Generate Onboarding token for consumer cluster via following steps:

1. Download ticketgen.sh script from:

   https://raw.githubusercontent.com/jarrpa/ocs-operator/ticketgen/hack/ticketgen/ticketgen.sh

2. Save private key from AUTH[“managed_service”][“private_key”] to

   temporary file.

3. Run ticketgen.sh script to generate Onboarding token.

Raises:

• CommandFailed – In case the script ticketgen.sh fails.

• ConfigurationError – when AUTH[“managed_service”][“private_key”] not is not defined

Returns:

Onboarding token

Return type:

string

ocs_ci.utility.managedservice.get_storage_provider_endpoint(cluster)

Get get_storage_provider_endpoint

Parameters:

cluster (str) – cluster name

Returns:

value of storage provider endpoint

Return type:

str

ocs_ci.utility.managedservice.remove_header_footer_from_key(key)

This function will remove header and footer from key (like: —–BEGIN RSA PRIVATE KEY—– —–END RSA PRIVATE KEY—– ) and return the key on one line.

Returns:

one line key string without header and footer

Return type:

string

ocs_ci.utility.memory module

Module for memory related util functions.

class ocs_ci.utility.memory.MemoryMonitor(interval, function, args=None, kwargs=None)

Bases: Timer

class to implement threading.Timer running func in a background

run()

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

ocs_ci.utility.memory.catch_empty_mem_df(df: DataFrame)

routine function to catch psutil failures and fill memory dataframe with failure markers, therefore we may see number of failures and ignore them on examination stage

ocs_ci.utility.memory.get_consumed_ram(proc: Process = psutil.Process(pid=1103, name='python', status='running', started='17:25:34'))

Get consumed RAM(rss) for the process rss is the Resident Set Size, which is the actual physical memory the process is using

ocs_ci.utility.memory.get_consumed_virt_mem(proc: Process = psutil.Process(pid=1103, name='python', status='running', started='17:25:34'))

Get consumed Virtual mem for the process vms is the Virtual Memory Size which is the virtual memory that process is using

! Important notice ! ‘vms’ will be larger than physical memory capacity: Shared libraries and frameworks are counted as part of the virtual memory for every application that uses them, e.g. if you have 100 processes running on a computer, and a 5 MB library used by all those processes, then that library is counted as 500 MB of virtual memory.

ocs_ci.utility.memory.get_peak_sum_mem() → tuple

get peak summarized memory stats for the test. Each test df file created anew. spikes defined per measurment (once in three seconds by default -> start_monitor_memory())

ocs_ci.utility.memory.mon: MemoryMonitor

ocs_ci.utility.memory.peak_mem_stats_human_readable(stat: <module 'ocs_ci.ocs.constants' from '/home/docs/checkouts/readthedocs.org/user_builds/ocs-ci/checkouts/latest/ocs_ci/ocs/constants.py'>, csv_path: str | None = None) → DataFrame

make peak mem stats dataframe human-readable dataframe columns = [name, proc_start, proc_end, rss_peak]

Parameters:

• stat (constants) – stat either ‘rss’ or ‘vms’ (constants.RAM | constants.VIRT)

• csv_path (str) – path to csv file with structure: index,pid,name,ts,rss,vms,status; will be ignored in case if df != None

Returns:

peak memory stats dataframe

Return type:

pd.DataFrame

ocs_ci.utility.memory.read_peak_mem_stats(stat: <module 'ocs_ci.ocs.constants' from '/home/docs/checkouts/readthedocs.org/user_builds/ocs-ci/checkouts/latest/ocs_ci/ocs/constants.py'>, df: ~pandas.core.frame.DataFrame | None = None, csv_path: str | None = None) → DataFrame

Read peak memory stats from Dataframe or csv file. Processes with stat above avg will be taken Table will be reduced to only processes with stat > avg(stat) if number of processes will be larger than 10

Parameters:

• stat (constants) – stat either ‘rss’ or ‘vms’ (constants.RAM | constants.VIRT)

• df (pd.DataFrame) – dataframe object with structure: index,pid,name,ts,rss,vms,status

• csv_path (str) – path to csv file with structure: index,pid,name,ts,rss,vms,status; will be ignored in case if df != None

Returns: pd.DataFrame similar to: name proc_start proc_end rss_peak 0 Google Chrome 2022-12-23 14:25:36 2022-12-23 14:27:32 156 MB 1 Google Chrome Helper (Renderer) 2022-12-23 14:25:39 2022-12-23 14:27:32 784 MB 2 Python 2022-12-23 14:25:22 2022-12-23 14:27:32 228 MB

ocs_ci.utility.memory.start_monitor_memory(interval: int = 3, create_csv: bool = False) → MemoryMonitor

Start memory monitor Timer process

Parameters:

• interval (int) – interval in sec to read measurements. Min interval is 2 sec

• create_csv (bool) – create csv during test run. With this option it is possible to upload file as artifact (to be done) or preserve csv file in the system

Returns:

monitor object MemoryMonitor(Timer)

Return type:

MemoryMonitor

ocs_ci.utility.memory.stop_monitor_memory(save_csv: bool = False) → tuple

Stop MemoryMonitor(Timer) and read memory stats

Parameters:

save_csv – saves csv temporarily, until main process is dead if save_csv = True; require create_csv=True at start_monitor_memory(…)

ocs_ci.utility.metadata_utils module

Module that contains all operations related to add metadata feature in a cluster

ocs_ci.utility.metadata_utils.available_subvolumes(sc_name, toolbox_pod, fs)

To fetch available subvolumes for cephfs or rbd

Parameters:

• sc_name (str) – storage class

• toolbox_pod (str) – ceph tool box pod

• fs (str) – file system

Returns:

subvolumes available for rbd or cephfs

Return type:

list

ocs_ci.utility.metadata_utils.check_setmetadata_availability(pod_obj)

Check setmetadata parameter is available or not for cephfs and rbd plugin pods

Parameters:

pod_obj (obj) – pod object

ocs_ci.utility.metadata_utils.created_subvolume(available_subvolumes, updated_subvolumes, sc_name)

To fetch created subvolume for cephfs or rbd

Parameters:

• available_subvolumes (list) – List of available subvolumes

• updated_subvolumes (list) – Updated list of subvolumes

• sc_name (str) – storage class

Returns:

name of subvolume created

Return type:

str

ocs_ci.utility.metadata_utils.disable_metadata(config_map_obj, pod_obj)

Disable CSI_ENABLE_METADATA

Parameters:

• config_map_obj (obj) – configmap object

• pod_obj (obj) – pod object

Steps: 1:- Disable CSI_ENABLE_METADATA flag via patch request 2:- Check csi-cephfsplugin provisioner and csi-rbdplugin-provisioner pods are up and running 3:- Check ‘setmatadata’ is not set for csi-cephfsplugin-provisioner and csi-rbdplugin-provisioner pods

ocs_ci.utility.metadata_utils.enable_metadata(config_map_obj, pod_obj)

Enable CSI_ENABLE_METADATA

Parameters:

• config_map_obj (obj) – configmap object

• pod_obj (obj) – pod object

Steps: 1:- Enable CSI_ENABLE_METADATA flag via patch request 2:- Check csi-cephfsplugin provisioner and csi-rbdplugin-provisioner pods are up and running 3:- Check ‘setmatadata’ is set for csi-cephfsplugin-provisioner and csi-rbdplugin-provisioner pods

Returns:

cluster name

Return type:

str

ocs_ci.utility.metadata_utils.fetch_metadata(sc_name, fs, toolbox_pod, created_subvol, snapshot=False, available_subvolumes=None, updated_subvolumes=None)

To fetch metadata details created for cephfs or rbd

Parameters:

• sc_name (str) – storage class

• toolbox_pod (str) – ceph tool box pod

• fs (str) – file system

• created_subvol (str) – Created sub volume

• snapshot (bool) – snapshot or not

• available_subvolumes (list) – List of available subvolumes

• updated_subvolumes (list) – Updated list of subvolumes

Returns:

metadata details

Return type:

json

ocs_ci.utility.metadata_utils.update_testdata_for_external_modes(sc_name, fs, external_mode=False)

Update the file sytem and storage class names for external mode clusters

Parameters:

• sc_name (str) – storage class

• fs (str) – file system

• external_mode (bool) – External mode or not

Returns:

storage class fs (str): file system

Return type:

sc_name (str)

ocs_ci.utility.metadata_utils.validate_metadata(metadata, clustername, pv_name=None, pvc_name=None, namespace=None, volumesnapshot_name=None, volumesnapshot_content=None)

To validate the metadata details

Parameters:

• metadata (json) – metadata details

• clustername (str) – cluster name

• pv_name (str) – name of the pv

• pvc_name (str) – name of the pvc

• namespace (str) – namespace

• volumesnapshot_name (str) – name of the volumesnapshot

• volumesnapshot_content (str) – volumesnapshot content

ocs_ci.utility.mirror_openshift module

ocs_ci.utility.mirror_openshift.prepare_mirror_openshift_credential_files()

This function is a context manager.

Prepare local files containing username and password for mirror.openshift.com:

data/mirror_openshift_user, data/mirror_openshift_password.

Those files are used as variable files for repo files (stored in /etc/yum/vars/ on the target RHEL nodes or pods).

Returns:

tuple containing two strings - file names for username file and password file

Return type:

(tuple)

Raises:

ConfigurationError – if mirror_openshift credentials are not provided in auth.yaml file

ocs_ci.utility.monitoring_tool module

ocs_ci.utility.monitoring_tool.check_go_version()

Check installed ‘go’ version

ocs_ci.utility.monitoring_tool.comparetool_deviation_check(first_file, second_file, deviation_list)

Function to run ‘comparetool’ and verify deviation_list accordingly to comparetool output

Parameters:

• first_file (Path) – path to the file, standing first in comparetool args to compare

• second_file (Path) – path to the file, standing second in comparetool args to compare

• deviation_list (list) – list of deviation strings expected to be in comparetool output

ocs_ci.utility.monitoring_tool.convert_yaml_file_to_json_file(file_path)

Util to convert yaml file to json file and replace an ext of the files

Parameters:

file_path (Path) – path to the file to convert

Returns:

path to the new file, yaml converted to json

Return type:

Path

ocs_ci.utility.nfs_utils module

Module that contains all operations related to nfs feature in a cluster

ocs_ci.utility.nfs_utils.create_nfs_load_balancer_service(storage_cluster_obj)

Create the nfs loadbalancer service

Parameters:

storage_cluster_obj (obj) – storage cluster object

ocs_ci.utility.nfs_utils.delete_nfs_load_balancer_service(storage_cluster_obj)

Delete the nfs loadbalancer service

Parameters:

storage_cluster_obj (obj) – storage cluster object

ocs_ci.utility.nfs_utils.nfs_disable(storage_cluster_obj, config_map_obj, pod_obj, sc, nfs_ganesha_pod_name)

Disable nfs feature and ROOK_CSI_ENABLE_NFS Steps: 1:- Disable nfs feature for storage-cluster 2:- Disable ROOK_CSI_ENABLE_NFS via patch request 3:- Delete CephNFS, ocs nfs Service and the nfs StorageClass 4:- Wait untill nfs-ganesha pod deleted

Parameters:

• storage_cluster_obj (obj) – storage cluster object

• config_map_obj (obj) – config map object

• pod_obj (obj) – pod object

• sc (str) – nfs storage class

• nfs_ganesha_pod_name (str) – rook-ceph-nfs * pod name

ocs_ci.utility.nfs_utils.nfs_enable(storage_cluster_obj, config_map_obj, pod_obj, namespace)

Enable nfs feature and ROOK_CSI_ENABLE_NFS Steps: 1:- Enable nfs feature for storage-cluster 2:- Enable ROOK_CSI_ENABLE_NFS via patch request 3:- Check nfs-ganesha server is up and running 4:- Check csi-nfsplugin pods are up and running

Parameters:

• storage_cluster_obj (obj) – storage cluster object

• config_map_obj (obj) – config map object

• pod_obj (obj) – pod object

• namespace (str) – namespace name

Returns:

nfs-ganesha pod name

Return type:

str

ocs_ci.utility.nfs_utils.skip_test_if_nfs_client_unavailable(nfs_client_ip)

Skip the tests if a valid nfs client ip is not available for outside mounts

Parameters:

nfs_client_ip (str) – nfs client ip address

ocs_ci.utility.nfs_utils.unmount(con, test_folder)

Unmount existing mount points

Parameters:

• con (obj) – connection object

• test_folder (str) – Mount path

ocs_ci.utility.ocp_upgrade module

ocs_ci.utility.ocp_upgrade.pause_machinehealthcheck()

During the upgrade process, nodes in the cluster might become temporarily unavailable. In the case of worker nodes, the machine health check might identify such nodes as unhealthy and reboot them. To avoid rebooting such nodes, pause all the MachineHealthCheck resources before updating the cluster. This step is based on OCP documentation for OCP 4.9 and above.

ocs_ci.utility.ocp_upgrade.resume_machinehealthcheck()

Resume the machine health checks after updating the cluster. To resume the check, remove the pause annotation from the MachineHealthCheck resource.

ocs_ci.utility.ocs_build module

This module is used to return latest OCS internal build for specified OCS version.

ocs_ci.utility.ocs_build.init_arg_parser()

Init argument parser.

Returns:

Parsed arguments

Return type:

object

ocs_ci.utility.ocs_build.main()

Main function

ocs_ci.utility.openshift_console module

class ocs_ci.utility.openshift_console.OpenshiftConsole(browser='chrome')

Bases: object

run_openshift_console(suite, env_vars=None, timeout=1500, log_suffix='')

Run openshift console suite

Parameters:

• suite (str) – openshift console suite to execute

• env_vars (dict) – env variables to expose for openshift console

• timeout (int) – timeout for test-gui.sh script

Raises:

OpenshiftConsoleSuiteNotDefined – if suite is not defined

setup_console_prereq()

Setup openshift console prerequisites

Raises:

UnsupportedBrowser – in case the browser is not supported

ocs_ci.utility.openshift_dedicated module

Module for interactions with Openshift Dedciated Cluster.

ocs_ci.utility.openshift_dedicated.create_cluster(cluster_name)

Create OCP cluster.

Parameters:

cluster_name (str) – Cluster name.

ocs_ci.utility.openshift_dedicated.destroy_cluster(cluster)

Destroy the cluster on Openshift Dedicated.

Parameters:

cluster (str) – Cluster name or ID.

ocs_ci.utility.openshift_dedicated.download_ocm_cli()

Method to download OCM cli

Returns:

path to the installer

Return type:

str

ocs_ci.utility.openshift_dedicated.get_cluster_details(cluster)

Returns info about the cluster which is taken from the OCM command.

Parameters:

cluster (str) – Cluster name.

ocs_ci.utility.openshift_dedicated.get_credentials(cluster)

Get json with cluster credentials

Parameters:

cluster (str) – Cluster name.

Returns:

cluster credentials

Return type:

json

ocs_ci.utility.openshift_dedicated.get_kubeadmin_password(cluster, path)

Export password for kubeadmin to provided path.

Parameters:

• cluster (str) – Cluster name.

• path (str) – Path where to create kubeadmin-password file.

ocs_ci.utility.openshift_dedicated.get_kubeconfig(cluster, path)

Export kubeconfig to provided path.

Parameters:

• cluster (str) – Cluster name.

• path (str) – Path where to create kubeconfig file.

ocs_ci.utility.openshift_dedicated.list_cluster()

Returns info about the openshift dedciated clusters which is taken from the OCM command.

ocs_ci.utility.openshift_dedicated.login()

Login to OCM client

ocs_ci.utility.pagerduty module

class ocs_ci.utility.pagerduty.PagerDutyAPI(token=None)

Bases: object

This is wrapper class for PagerDuty API: https://developer.pagerduty.com/api-reference/

In order to use the API, there must be set api_key. That is taken from AUTH/pagerduty/api_key in config.

check_incident_cleared(summary, measure_end_time, time_min=420, pagerduty_service_ids=None)

Check that all incidents with provided summary are cleared.

Parameters:

• summary (str) – Incident summary

• measure_end_time (int) – Timestamp of measurement end

• time_min (int) – Number of seconds to wait for incidents to be cleared since measurement end

• pagerduty_service_ids (list) – service ids used in incidents get query

create(resource, payload=None)

Post resources from PagerDuty API.

Parameters:

• resource (str) – Represents part of uri that specifies given resource

• payload (dict) – Provide parameters to POST API call.

Returns:

Response from PagerDuty api

Return type:

dict

delete(resource)

Delete resource from PagerDuty API.

Parameters:

resource (str) – Represents part of uri that specifies given resource

Returns:

Response from PagerDuty api

Return type:

dict

get(resource, payload=None)

Get resources from PagerDuty API.

Parameters:

• resource (str) – Represents part of uri that specifies given resource

• payload (dict) – Provide parameters to GET API call. e.g. for incidents resource this can be {“service_ids[]”: <id>, “since”: “2021-07-15T00:00:00Z”, “time_zone”: “UTC”}

Returns:

Response from PagerDuty api

Return type:

dict

get_escalation_policy_id(name)

Get account escalation policy from PagerDuty API by name.

Returns:

Escalation policy id

Return type:

str

get_integration_dict(vendor_name)

Get a structure prepared to be a payload for integration creation via API.

Parameters:

vendor_name (str) – Name of vendor that is used in integration

Returns:

Structure containing all data required to by PagerDuty API

in order to create an integration

Return type:

dict

get_service_dict()

Get a structure prepared to be a payload for service creation via API.

Returns:

Structure containing all data required to by PagerDuty API

in order to create a service

Return type:

dict

get_vendor_id(name)

Get id of vendor with provided name from PagerDuty API.

Parameters:

name (str) – Vendor name

Returns:

Vendor id

Return type:

str

wait_for_incident_cleared(summary, timeout=1200, sleep=5, pagerduty_service_ids=None)

Search for incident to be cleared.

Parameters:

• summary (str) – Incident summary

• sleep (int) – Number of seconds to sleep in between incidents search

• pagerduty_service_ids (list) – service ids used in incidents get query

Returns:

List of incident records

Return type:

list

ocs_ci.utility.pagerduty.check_incident_list(summary, urgency, incidents, status='triggered')

Check list of incidents that there are incidents with requested label in summary and specific urgency. If some incident is missing then this check fails.

Parameters:

• summary (str) – String that is part of incident summary

• urgency (str) – Incident urgency

• incidents (list) – List of incidents to check

• status (str) – Incident status

Returns:

List of incidents that match search requirements

Return type:

list

ocs_ci.utility.pagerduty.get_pagerduty_service_id()

Get the pagerduty service id in the current run

Returns:

The pagerduty service id in the current run

Return type:

str

ocs_ci.utility.pagerduty.set_pagerduty_faas_secret(integration_key)

Update managed-fusion-agent-config secret. This is valid only on Fusion aaS platform. managed-fusion-agent-config secret is expected to be present prior to the update.

Parameters:

integration_key (str) – Integration key taken from PagerDuty Prometheus integration

ocs_ci.utility.pagerduty.set_pagerduty_integration_secret(integration_key)

Update ocs-converged-pagerduty secret. This is valid only on ODF Managed Service. ocs-converged-pagerduty secret is expected to be present prior to the update.

Parameters:

integration_key (str) – Integration key taken from PagerDuty Prometheus integration

ocs_ci.utility.pgsql module

Module for interactions postgresql database.

class ocs_ci.utility.pgsql.PgsqlManager(username, password, keep_connection=False, database='postgres', host='localhost', port=5432)

Bases: object

class Cursor(conn)

Bases: object

Context manager which make sure the cursor is opened and closed.

close_conn()

Method to close the connection.

property conn

Connection property.

conn_closure()

Decorator which will make sure the connection is closed after call of method decorated with. This also depends on self.keep_connection which may not close the connection if set to True, but in case of some error or crash, it will make sure the connection is always properly closed.

create_database(db_name, extra_params)

Create database.

Parameters:

• db_name (string) – database name

• extra_params (string) – extra postgresql command to add after CREATE DATABASE db_name

delete_database(db_name)

Delete the database.

Parameters:

db_name (string) – database name

ocs_ci.utility.powernodes module

class ocs_ci.utility.powernodes.PowerNodes

Bases: object

Wrapper for PowerNodes

iskvm()

Verify if this environment is libvirt or kvm.

Returns:

True if this is kvm environment, False otherwise

Return type:

bool

restart_powernodes_machines(powernode_machines, timeout, wait, force=True)

Restart PowerNode Machines

Parameters:

• powernode_machines (list) – PowerNode objects

• timeout (int) – time in seconds to wait for node to reach ‘not ready’ state, and ‘ready’ state.

• wait (bool) – True if need to wait till the restarted node reaches timeout

• force (bool) – True for PowerNode ungraceful power off, False for graceful PowerNode shutdown - for future use

restart_powernodes_machines_powervs(powernode_machines, timeout, wait)

Restart PowerNode Machines

Parameters:

• powernode_machines (list) – PowerNode objects

• timeout (int) – time in seconds to wait for node to reach ‘not ready’ state, and ‘ready’ state.

• wait (bool) – True if need to wait till the restarted node reaches timeout

start_powernodes_machines(powernode_machines, timeout=900, wait=True, force=True)

Start PowerNode Machines

Parameters:

• powernode_machines (list) – List of PowerNode machines

• timeout (int) – time in seconds to wait for node to reach ‘not ready’ state, and ‘ready’ state.

• wait (bool) – Wait for PowerNodes to start - for future use

• force (bool) – True for PowerNode ungraceful power off, False for graceful PowerNode shutdown - for future use

start_powernodes_machines_powervs(powernode_machines, timeout=900, wait=True)

Start PowerNode Machines

Parameters:

• powernode_machines (list) – List of PowerNode machines

• timeout (int) – time in seconds to wait for node to reach ‘not ready’ state, and ‘ready’ state.

• wait (bool) – Wait for PowerNodes to start - for future use

stop_powernodes_machines(powernode_machines, timeout=900, wait=True, force=True)

Stop PowerNode Machines

Parameters:

• powernode_machines (list) – PowerNode objects

• timeout (int) – time in seconds to wait for node to reach ‘not ready’ state

• wait (bool) – True if need to wait till the restarted node reaches timeout - for future use

• force (bool) – True for PowerNode ungraceful power off, False for graceful PowerNode shutdown - for future use

Raises:

UnexpectedBehaviour – If PowerNode machine is still up

stop_powernodes_machines_powervs(powernode_machines, timeout=900, wait=True)

Stop PowerNode Machines - PowerVS

Parameters:

• powernode_machines (list) – PowerNodes objects

• timeout (int) – time in seconds to wait for node to reach ‘not ready’ state

• wait (bool) – True if need to wait till the restarted node reaches timeout - for future use

verify_machine_is_down(node)

Verify if PowerNode is completely powered off

Parameters:

node (object) – Node objects

Returns:

True if machine is down, False otherwise

Return type:

bool

ocs_ci.utility.prometheus module

class ocs_ci.utility.prometheus.PrometheusAPI(user=None, password=None, threading_lock=None)

Bases: object

This is wrapper class for Prometheus API.

check_alert_cleared(label, measure_end_time, time_min=120)

Check that all alerts with provided label are cleared.

Parameters:

• label (str) – Alerts label

• measure_end_time (int) – Timestamp of measurement end

• time_min (int) – Number of seconds to wait for alert to be cleared since measurement end

generate_cert()

Generate CA certificate from kubeconfig for API.

TODO: find proper way how to generate/load cert files.

get(resource, payload=None, timeout=300)

Get alerts from Prometheus API.

Parameters:

• resource (str) – Represents part of uri that specifies given resource

• payload (dict) – Provide parameters to GET API call. e.g. for alerts resource this can be {‘silenced’: False, ‘inhibited’: False}

• timeout (int) – Number of seconds to wait for Prometheus endpoint to get available if it is not available

Returns:

Response from Prometheus alerts api

Return type:

requests.models.Response

prometheus_log(prometheus_alert_list)

Log all alerts from Prometheus API to list

Parameters:

prometheus_alert_list (list) – List to be populated with alerts

query(query, timestamp=None, timeout=None, validate=True, mute_logs=False, log_debug=False)

Perform Prometheus instant query. This is a simple wrapper over get() method with plumbing code for instant queries, additional validation and logging.

Parameters:

• query (str) – Prometheus expression query string.

• timestamp (str) – Evaluation timestamp (rfc3339 or unix timestamp). Optional.

• timeout (str) – Evaluation timeout in duration format. Optional.

• validate (bool) – Perform basic validation on the response. Optional, True is the default. Use False when you expect query to fail eg. during negative testing.

• mute_logs (bool) – True for muting the logs, False otherwise

• log_debug (bool) – True for logging in debug, False otherwise

Returns:

Result of the query (value(s) for a single timestamp)

Return type:

list

query_range(query, start, end, step, timeout=None, validate=True)

Perform Prometheus range query. This is a simple wrapper over get() method with plumbing code for range queries, additional validation and logging.

Parameters:

• query (str) – Prometheus expression query string.

• start (str) – start timestamp (rfc3339 or unix timestamp)

• end (str) – end timestamp (rfc3339 or unix timestamp)

• step (float) – Query resolution step width as float number of seconds.

• timeout (str) – Evaluation timeout in duration format. Optional.

• validate (bool) – Perform basic validation on the response. Optional, True is the default. Use False when you expect query to fail eg. during negative testing.

Returns:

result of the query

Return type:

list

refresh_connection()

Login into OCP, refresh endpoint and token.

wait_for_alert(name, state=None, timeout=1200, sleep=5)

Search for alerts that have requested name and state.

Parameters:

• name (str) – Alert name

• state (str) – Alert state, if provided then there are searched alerts with provided state. If not provided then alerts are searched for absence of the alert. Loop that looks for alerts is broken when there are no alerts returned from API. This is done because API is not returning any alerts that are not in pending or firing state

• timeout (int) – Number of seconds for how long the alert should be searched

• sleep (int) – Number of seconds to sleep in between alert search

Returns:

List of alert records

Return type:

list

class ocs_ci.utility.prometheus.PrometheusAlertSubscriber(threading_lock, interval: float)

Bases: Timer

clear_alerts()

Clear alert list

get_alerts()

Get list of all alerts

prometheus_alert_list = []

run()

Run logging of all prometheus alerts.

! This method is called by Timer class, do not call it directly !

subscribe()

Start logging of all prometheus alerts

unsubscribe()

Stop logging of all prometheus alerts

ocs_ci.utility.prometheus.check_alert_list(label, msg, alerts, states, severity='warning', ignore_more_occurences=True)

Check list of alerts that there are alerts with requested label and message for each provided state. If some alert is missing then this check fails.

Parameters:

• label (str) – Alert label

• msg (str) – Alert message

• alerts (list) – List of alerts to check

• states (list) – List of states to check, order is important

• ignore_more_occurences (bool) – If true then there is checkced only occurence of alert with requested label, message and state but it is not checked if there is more of occurences than one.

ocs_ci.utility.prometheus.check_query_range_result_enum(result, good_values, bad_values=(), exp_metric_num=None, exp_delay=None, exp_good_time=None)

Check that result of range query matches given expectations. Useful for metrics which convey status (eg. ceph health, ceph_osd_up), so that you can assume that during a given period, it’s value should match given single (or tuple of) value(s).

Parameters:

• result (list) – Data from query_range() method.

• good_values (tuple) – Tuple of values considered good

• bad_values (tuple) – Tuple of values considered bad, indicating a problem (optional, use if you need to distinguish bad and invalid values)

• exp_metric_num (int) – expected number of data series in the result, optional (eg. for ceph_health_status this would be 1, but for something like ceph_osd_up this will be a number of OSDs in the cluster)

• exp_delay (int) – Number of seconds from the start of the query time range for which we should tolerate bad values. This is useful if you change cluster state and processing of this change is expected to take some time.

• exp_good_time (int) – Number of seconds during which we should see good values in the metrics data. When this time passess values can go bad (but can’t be invalid). If not specified, good values should be presend during the whole time.

Returns:

True if result matches given expectations, False otherwise

Return type:

bool

ocs_ci.utility.prometheus.check_query_range_result_limits(result, good_min, good_max, exp_metric_num=None, exp_delay=None, exp_good_time=None)

Check that result of range query matches given expectations. Useful for metrics which convey continuous value, eg. storage or cpu utilization.

Parameters:

• result (list) – Data from query_range() method.

• good_min (float) – Min. value which is considered good.

• good_max (float) – Max. value which is still considered as good.

• exp_metric_num (int) – expected number of data series in the result, optional (eg. for ceph_health_status this would be 1, but for something like ceph_osd_up this will be a number of OSDs in the cluster)

• exp_delay (int) – Number of seconds from the start of the query time range for which we should tolerate bad values. This is useful if you change cluster state and processing of this change is expected to take some time.

• exp_good_time (int) – Number of seconds during which we should see good values in the metrics data. When this time passess values can go bad (but can’t be invalid). If not specified, good values should be presend during the whole time.

Returns:

True if result matches given expectations, False otherwise

Return type:

bool

ocs_ci.utility.prometheus.check_query_range_result_viafunction(result, is_value_good, is_value_bad=<function <lambda>>, exp_metric_num=None, exp_delay=None, exp_good_time=None, is_float=False)

Check that result of range query matches expectations expressed via is_value_good (and optionally is_value_bad) functions, which takes a value and returns True if the value is good (or bad).

Parameters:

• result (list) – Data from query_range() method.

• is_value_good (function) – returns True for a good value

• is_value_bad (function) – returns True for a bad balue, indicating a problem (optional, use if you need to distinguish bad and invalid values)

• exp_metric_num (int) – expected number of data series in the result, optional (eg. for ceph_health_status this would be 1, but for something like ceph_osd_up this will be a number of OSDs in the cluster)

• exp_delay (int) – Number of seconds from the start of the query time range for which we should tolerate bad values. This is useful if you change cluster state and processing of this change is expected to take some time.

• exp_good_time (int) – Number of seconds during which we should see good values in the metrics data. When this time passess values can go bad (but can’t be invalid). If not specified, good values should be presend during the whole time.

• is_float (bool) – assume that the value is float, otherwise assume int

Returns:

True if result matches given expectations, False otherwise

Return type:

bool

ocs_ci.utility.prometheus.log_parsing_error(query, resp_content, ex)

Log an error raised during parsing of a prometheus query.

Parameters:

• query (dict) – Full specification of a prometheus query.

• resp_content (bytes) – Response from prometheus

• ex (Exception) – Exception raised during parsing of prometheus reply.

ocs_ci.utility.prometheus.validate_status(content)

Validate content data from Prometheus. If this fails, Prometheus instance or a query is so broken that test can’t be performed. We assume that Prometheus reports “success” even for queries which returns nothing.

Parameters:

content (dict) – data from Prometheus

Raises:

• TypeError – when content is not a dict

• ValueError – when status of the content is not success

ocs_ci.utility.proxy module

ocs_ci.utility.proxy.update_kubeconfig_with_proxy_url_for_client(kubeconfig)

If it is a proxy environment and client_http_proxy is defined, update each cluster configuration in kubeconfig with proxy-url parameter to redirect client access through proxy server

Parameters:

kubeconfig (str) – path to kubeconfig file

ocs_ci.utility.psiutils module

A module for all PSI-openstack related utilities

class ocs_ci.utility.psiutils.PSIUtils(psi_conf)

Bases: object

A class for handling PSI functionalities

attach_volume(vol, instance_id)

Attach the given volume to the specific PSI openstack instance

Parameters:

• vol (Volume) – cinder volume object

• instance_id (str) – uuid of the instance

check_expected_vol_status(vol, expected_state)

Check status of the volume and return true if it matches the expected state

Parameters:

• vol (cinderclient.volume) – Volume object for which state needs to be checked

• expected_state (str) – Expected state of the volume

Returns:

True if state is same as expected else False

Return type:

bool

check_vol_deleted(vol)

Check whether its delete

Parameters:

vol (cinderclient.Volume) – volume object

Returns:

True if deleted else False

Return type:

bool

create_volume(name, size=100, volume_type='tripleo', metadata=None)

A function to create openstack volumes

Parameters:

• name (str) – display name of the volume

• size (int) – of the volume in GBs

• volume_type (str) – type of volume to be created

• metadata (dict) – Any {k,v} to be associated with volume

Returns:

cinderclient.Client.Volumes

Return type:

Volume

detach_and_delete_vols(volumes)

Detach and delete volumes from the list

Parameters:

volumes (list) – of Volume objects

get_instances_with_pattern(pattern)

Get instances matching pattern

Parameters:

pattern (str) – Pattern for matching instance name note: we are looking for only ‘name’ param for pattern matching

Returns:

novaclient.base.ListWithMeta

get_volumes_with_tag(tag)

Get PSI volumes having this tag (k,v)

Parameters:

tag (dict) – of desired (k,v)

Returns:

of cinderclient.<vers>.volumes object

Return type:

list

ocs_ci.utility.reporting module

ocs_ci.utility.reporting.get_polarion_id(upgrade=False)

Determine the polarion_id of the deployment or upgrade

Parameters:

upgrade (bool) – get upgrade_id if true, else get deployment_id

Returns:

polarion_id of the deployment or upgrade

Return type:

str

ocs_ci.utility.reporting.get_rp_launch_attributes()

Retrieve information from the config to use as launch attributes in ReportPortal.

Returns:

ReportPortal launch attributes

Return type:

dict

ocs_ci.utility.reporting.get_rp_launch_description()

Construct and return the ReportPortal launch description.

Returns:

ReportPortal launch description

Return type:

(str)

ocs_ci.utility.reporting.get_rp_launch_name()

Construct and return the ReportPortal launch name :returns: ReportPortal launch name :rtype: str

ocs_ci.utility.reporting.update_live_must_gather_image()

Update live must gather image in the config.

ocs_ci.utility.retry module

ocs_ci.utility.retry.retry(exception_to_check, tries=4, delay=3, backoff=2, text_in_exception=None, func=None)

Retry calling the decorated function using exponential backoff.

Parameters:

• exception_to_check – the exception to check. may be a tuple of exceptions to check

• tries – number of times to try (not retry) before giving up

• delay – initial delay between retries in seconds

• backoff – backoff multiplier e.g. value of 2 will double the delay each retry

• text_in_exception – Retry only when text_in_exception is in the text of exception

• func – function for garbage collector

ocs_ci.utility.rgwutils module

ocs_ci.utility.rgwutils.get_rgw_count(ocs_version, is_upgrade, version_before_upgrade)

Get RGW Count

RGW Count is 2 if:

OCS 4.5 unless upgraded from a prior version OCS 4.6 arbiter deployment

Otherwise, RGW Count is 1

Parameters:

• ocs_version (str, float) – OCS Version

• is_upgrade (bool) – If cluster was upgraded to current version

• version_before_upgrade (str, float) – OCS Version prior to upgrade

Returns:

RGW Count

Return type:

int

ocs_ci.utility.rhv module

Module for interactions with RHV Cluster

class ocs_ci.utility.rhv.RHV(url=None, username=None, password=None, insecure=True, ca_file=None)

Bases: object

Wrapper for RHV

add_disk(vm, size, disk_format='RAW', disk_interface='VIRTIO_SCSI', sparse=None, pass_discard=None, storage_domain_id=None, timeout=120)

Attaches disk to VM

Parameters:

• vm (types.Vm) – Vm instance

• size (int) – size of disk in GB

• disk_format (str) – underlying storage format of disks (default: “RAW”)

• disk_interface (str) – underlying storage interface of disks communication with controller (default: ‘VIRTIO_SCSI’)

• sparse (bool) – disk allocation policy. True for sparse, false for preallocated (default: None)

• pass_discard (bool) – True if the virtual machine passes discard commands to the storage, False otherwise (default: None)

• storage_domain_id (str) – A unique identifier for the storage domain

• timeout (int) – The timeout in seconds for disk status OK (default: 120)

add_disks(num_disks, vm, size, disk_format='RAW', disk_interface='VIRTIO_SCSI', sparse=None, pass_discard=None, storage_domain_id=None, timeout=120)

Adds multiple disks to the VM

Parameters:

• num_disks – number of disks to add

• vm (types.Vm) – Vm instance

• size (int) – size of disk in GB

• disk_format (str) – underlying storage format of disks (default: “RAW”)

• disk_interface (str) – underlying storage interface of disks communication with controller (default: ‘VIRTIO_SCSI’)

• sparse (bool) – disk allocation policy. True for sparse, false for preallocated (default: None)

• pass_discard (bool) – True if the virtual machine passes discard commands to the storage, False otherwise (default: None)

• storage_domain_id (str) – A unique identifier for the storage domain

• timeout (int) – The timeout in seconds for disk status OK (default: 120)

close_engine_connection(connection)

Releases the resources used by this connection.

create_engine_connection()

Create the connection to the engine server

Returns:

Connection to the engine server

Return type:

connection

property disks_service

Reference to the disks service

property engine_connection

Get the connection to the engine server

Returns:

Connection to the engine server

Return type:

connection

get_compute_vms()

Gets all compute VM’s

Returns:

list of compute Vm instances

Return type:

list

get_disk_attachment_service(vm_id, disk_id)

Get the service that manages a specific disk attachment

Parameters:

• vm_id (str) – unique identifier of the vm

• disk_id (str) – unique identifier of the disk

get_disk_attachments(vm_id)

Get the disks that are attached to the virtual machine

Parameters:

vm_id (str) – unique identifier of the vm

Returns:

list of disks attachments

Return type:

list

get_disk_attachments_service(vm_id)

Get the service that manages the disk attachments of the vm

Parameters:

vm_id (str) – unique identifier of the vm

get_disk_service(disk_id)

Get the disk service managing a specific disk

Parameters:

disk_id (str) – unique identifier of the disk

get_disks_by_pattern(pattern=None, case_sensitive=True, key='name', filter_by_cluster_name=True)

Get the list of disks by pattern. If not specified it will return all the disks, or will return the list filtered by the cluster name.

Parameters:

• pattern (str) – query string to restrict the returned disks

• case_sensitive (bool) – True to take case into account, False to ignore case (default: True)

• key (str) – Either ‘name’, or ‘id’ (default: name)

• filter_by_cluster_name (bool) – Will be used only if the ‘pattern’ param not specified. If True it filters the disks by the cluster name, else if False it returns all disks.

Returns:

list of disks

Return type:

list

get_rhv_vm_instance(vm_name)

Get RHV VM instance

Parameters:

vm_name – name of RHV VM

Returns:

Vm instance

Return type:

vm (types.Vm)

get_vm_names()

Returns:

list of VMs name

Return type:

list

get_vm_service(vm_id)

Get the service that manages a specific virtual machine

Parameters:

vm_id (str) – unique identifier of the vm

get_vm_status(vm)

Get the power status of RHV VM

Parameters:

vm (str) – RHV VM instance

Returns :

str: Power status of RHV VM

get_vms_by_pattern(pattern=None, case_sensitive=True, key='name', filter_by_cluster_name=True)

Get the list of virtual machines of the system by pattern. If not specified it will return all the vms, or will return the list filtered by the cluster name.

Parameters:

• pattern (str) – query string to restrict the returned vms

• case_sensitive (bool) – True to take case into account, False to ignore case (default: True)

• key (str) – Either ‘name’, or ‘id’ (default: name)

• filter_by_cluster_name (bool) – Will be used only if the ‘pattern’ param not specified. If True it filters the Vm by the cluster name, else if False it returns all Vm.

Returns:

list of Vm instances

Return type:

list

reboot_rhv_vms(vms, timeout=600, wait=True, force=True)

Reboot the RHV virtual machines

Parameters:

• vms (list) – Names of RHV vms

• timeout (int) – time in seconds to wait for VM to reboot

• wait (bool) – Wait for RHV VMs to reboot

• force (bool) – True to reboot vm forcibly False otherwise

remove_disk(vm, identifier, key='name', detach_only=True)

Removes the disk attachment. By default, only detach the disk from the virtual machine, but won’t remove it from the system, unless the detach_only parameter is False

Parameters:

• vm (types.Vm) – Vm instance

• identifier (str) – unique identifier of the disk Either disk name or disk id

• key (str) – Either ‘name’ or ‘id’ (default: name)

• detach_only (bool) – True to only detach the disk from the vm but not removed from the system, False otherwise (default: True)

remove_disks(vm, detach_only=True)

Removes all the extra disks for a VM

Parameters:

• vm (types.Vm) – Vm instance

• detach_only (bool) – True to only detach the disk from the vm but not removed from the system, False otherwise (default: True)

restart_rhv_vms_by_stop_and_start(vms, wait=True, force=False)

Stop and Start RHV virtual machines

Parameters:

• vms (list) – list of RHV vm instances

• force (bool) – True for non-graceful VM shutdown, False for graceful VM shutdown.

• wait (bool) – Wait for RHV VMs to start

start_rhv_vms(vms, wait=True, timeout=600)

Run the RHV virtual machines

Parameters:

• vms (list) – list of RHV vm instances

• wait (bool) – Wait for RHV VMs to start

• timeout (int) – time in seconds to wait for VM to reach ‘up’ status.

stop_rhv_vms(vms, timeout=600, force=False)

Shutdown the RHV virtual machines

Parameters:

• vms (list) – list of RHV vm instances

• force (bool) – True for non-graceful VM shutdown, False for graceful VM shutdown.

• timeout (int) – time in seconds to wait for VM to reach ‘down’ status.

property system_service

Reference to the root of the services tree

The returned value is an instance of the SystemService class

property vms_service

Reference to the virtual machines service

ocs_ci.utility.rosa module

Module for interactions with Openshift Dedciated Cluster.

ocs_ci.utility.rosa.appliance_mode_cluster(cluster_name)

Create appliance mode provider cluster

Parameters:

cluster_name (str) – Cluster name

ocs_ci.utility.rosa.create_account_roles(prefix='ManagedOpenShift')

Create the required account-wide roles and policies, including Operator policies.

Parameters:

prefix (str) – role prefix

ocs_ci.utility.rosa.create_cluster(cluster_name, version, region)

Create OCP cluster.

Parameters:

• cluster_name (str) – Cluster name

• version (str) – cluster version

• region (str) – Cluster region

ocs_ci.utility.rosa.create_oidc_provider(cluster)

Create the OpenID Connect (OIDC) provider that the Operators will use to authenticate

Parameters:

cluster (str) – cluster name or cluster id

ocs_ci.utility.rosa.create_operator_roles(cluster)

Create the cluster-specific Operator IAM roles. The roles created include the relevant prefix for the cluster name

Parameters:

cluster (str) – cluster name or cluster id

ocs_ci.utility.rosa.delete_odf_addon(cluster)

Delete ODF Managed Service addon from cluster.

Parameters:

cluster (str) – cluster name or cluster id

ocs_ci.utility.rosa.delete_oidc_provider(cluster_id)

Delete oidc provider of the given cluster

Parameters:

cluster_id (str) – the id of the cluster

ocs_ci.utility.rosa.delete_operator_roles(cluster_id)

Delete operator roles of the given cluster

Parameters:

cluster_id (str) – the id of the cluster

ocs_ci.utility.rosa.destroy_appliance_mode_cluster(cluster)

Delete rosa cluster if appliance mode

Parameters:

cluster – name of the cluster

Returns:

True if appliance mode and cluster delete initiated

else False

Return type:

bool

ocs_ci.utility.rosa.download_rosa_cli()

Method to download OCM cli

Returns:

path to the installer

Return type:

str

ocs_ci.utility.rosa.edit_addon_installation(addon_param_key, addon_param_value, cluster_name=None, addon_name=None, wait=True)

Edit a specific parameter of the odf addon installation of a cluster

Parameters:

• addon_param_key (str) – The addon key param to modify

• addon_param_value (str) – The addon value param to modify

• cluster_name (str) – The cluster name. The default value is ‘config.ENV_DATA[“cluster_name”]’

• addon_name (str) – The addon name. The default value is ‘addon_name = config.ENV_DATA[“addon_name”]’

• wait (bool) – If true, wait for the addon to be ready. False, otherwise.

ocs_ci.utility.rosa.get_addon_info(cluster, addon_name)

Get line related to addon from rosa list addons command.

Parameters:

• cluster (str) – cluster name

• addon_name (str) – addon name

Returns:

line of the command for relevant addon. If not found, it returns None.

Return type:

str

ocs_ci.utility.rosa.get_latest_rosa_version(version)

Returns latest available z-stream version available for ROSA.

Parameters:

version (str) – OCP version in format x.y

Returns:

Latest available z-stream version

Return type:

str

ocs_ci.utility.rosa.get_rosa_cluster_service_id(cluster)

Get service id of cluster

Parameters:

cluster (str) – cluster name

Returns:

service id of cluster. If not found, it returns None.

Return type:

str

ocs_ci.utility.rosa.get_rosa_service_details(cluster)

Returns info about the rosa service cluster.

Parameters:

cluster (str) – Cluster name.

ocs_ci.utility.rosa.install_odf_addon(cluster)

Install ODF Managed Service addon to cluster.

Parameters:

cluster (str) – cluster name or cluster id

ocs_ci.utility.rosa.is_odf_addon_installed(cluster_name=None)

Check if the odf addon is installed

Parameters:

cluster_name (str) – The cluster name. The default value is ‘config.ENV_DATA[“cluster_name”]’

Returns:

True, if the odf addon is installed. False, otherwise

Return type:

bool

ocs_ci.utility.rosa.login()

Login to ROSA client

ocs_ci.utility.rosa.post_onboarding_verification()

Check that after onboarding consumer the relevant resources were created in the provider cluster and then switch back to the original cluster: 1. StorageConsumer with the correct id exists and all its CephResources are in Ready status 2. CephBlockPool and Subvolumegroup with the correct id are in Ready status

ocs_ci.utility.rosa.validate_ocp_version(version)

Validate the version whether given version is z-stream version available for ROSA.

Parameters:

version (str) – OCP version string

Returns:

True if given version is available in z-stream version for ROSA

else False

Return type:

bool

ocs_ci.utility.rosa.wait_for_addon_to_be_ready(cluster_name=None, addon_name=None, timeout=300, sleep=20)

Wait for the addon of a cluster to be ready

Parameters:

• cluster_name (str) – The cluster name. The default value is ‘config.ENV_DATA[“cluster_name”]’

• addon_name (str) – The addon name. The default value is ‘addon_name = config.ENV_DATA[“addon_name”]’

• timeout (int) – Timeout to wait for the addon to be ready

• sleep (int) – Time in seconds to sleep between attempts

Raises:

TimeoutExpiredError – In case the addon is not ready in the given timeout

ocs_ci.utility.secret module

ocs_ci.utility.secret.link_all_sa_and_secret(secret_name, namespace)

Link all service accounts in specified namespace with the secret for pulling of images.

Parameters:

• secret_name (str) – secret name

• namespace (str) – namespace name

ocs_ci.utility.secret.link_all_sa_and_secret_and_delete_pods(secret_name, namespace)

Link all service accounts in specified namespace with the secret for pulling of images.

Parameters:

• secret_name (str) – secret name

• namespace (str) – namespace name

ocs_ci.utility.secret.link_sa_and_secret(sa_name, secret_name, namespace)

Link service account and secret for pulling of images.

Parameters:

• sa_name (str) – service account name

• secret_name (str) – secret name

• namespace (str) – namespace name

ocs_ci.utility.service module

class ocs_ci.utility.service.KubeletService

Bases: Service

Kubelet Service class

The purpose of this class is to extend Service class to provide stop/start/restart etc operations on kubelet service. Since kubelet service stop and start operations involve changing status of OCP node objects, this class verifies the same.

restart(node, timeout)

Restart the kubelet service using parent service class. After that, ensures the corresponding OCP node is connectable and moves to Ready state.

Parameters:

• node (object) – Node objects

• timeout (int) – time in seconds to wait for service to stop.

start(node, timeout)

Start the kubelet service using parent service class. After that, ensures the corresponding OCP node is connectable and moves to Ready state.

Parameters:

• node (object) – Node objects

• timeout (int) – time in seconds to wait for service to stop.

stop(node, timeout)

Stop the kubelet service using parent service class. After that, ensures the corresponding OCP node moves to NotReady state.

Parameters:

• node (object) – Node objects

• timeout (int) – time in seconds to wait for service to stop.

class ocs_ci.utility.service.Service(service_name, force=True)

Bases: object

Generic Service class

The purpose of this class is to provide a support to perform start/stop/restart and status operations on a given service. The class is instantiated with required service name without ‘.service extension. Refer to KubeletService class in this module for an example on how to use these methods.

kill(node, timeout)

Kill the given service using systemctl.

Parameters:

• node (object) – Node objects

• timeout (int) – time in seconds to wait for service to be stopped.

restart(node, timeout)

Restart the given service using systemctl.

Parameters:

• node (object) – Node objects

• timeout (int) – time in seconds to wait for service to be started.

start(node, timeout)

Start the given service using systemctl.

Parameters:

• node (object) – Node objects

• timeout (int) – time in seconds to wait for service to start.

Raises:

UnexpectedBehaviour – If service on powerNode machine is still not up

status(node, timeout)

Get the status of the given service using systemctl.

Parameters:

• node (object) – Node objects

• timeout (int) – Future use.

Returns:

‘active’ or ‘inactive’ or ‘failed’, etc.

Return type:

(string)

stop(node, timeout)

Stop the given service using systemctl.

Parameters:

• node (object) – Node objects

• timeout (int) – time in seconds to wait for service to stop.

Raises:

UnexpectedBehaviour – If service on PowerNode machine is still up

verify_service(node, action)

Verify if PowerNode is completely powered off

Parameters:

• node (object) – Node objects

• action (string) – ACTIVE or INACTIVE or FAILED

Returns:

True if service state is reqested action, False otherwise

Return type:

bool

ocs_ci.utility.ssl_certs module

This module is used for generating custom SSL certificates.

class ocs_ci.utility.ssl_certs.CSR

Bases: X509Req

Wrapper class over crypto.X509Req used for customization.

class ocs_ci.utility.ssl_certs.Key

Bases: PKey

Wrapper class over crypto.PKey used for customization.

class ocs_ci.utility.ssl_certs.OCSCertificate(signing_service, cn, sans=None)

Bases: object

Generate custom certificate signed by the automatic siging certification authority

Parameters:

• signing_service (str) – URL of the automatic signing CA service

• cn (str) – Certificate Common Name

• sans (list) – list of Subject Alternative Names (prefixed by the type like ‘DNS:’ or ‘IP:’)

property crt

property csr

generate_csr()

Generate Certificate Signing Request for the certificate

generate_key()

Generate private key for the certificate

get_crt()

Use automatic signing CA service to sign the CSR

property key

save_crt(path)

Save certificate to file

Parameters:

path (str) – path where to save certificate

save_csr(path)

Save certificate signing request to file

Parameters:

path (str) – path where to save certificate signing request

save_key(path)

Save certificate key to file

Parameters:

path (str) – path where to save certificate key

ocs_ci.utility.ssl_certs.configure_custom_api_cert(skip_tls_verify=False, wait_for_machineconfigpool=True)

Configure custom SSL certificate for API. If the certificate doesn’t exists, generate new one signed by automatic certificate signing service.

Parameters:

• skip_tls_verify (bool) – True if allow skipping TLS verification

• wait_for_machineconfigpool (bool) – True if it should wait for machineConfigPool

Raises:

ConfigurationError – when some required parameter is not configured

ocs_ci.utility.ssl_certs.configure_custom_ingress_cert(skip_tls_verify=False, wait_for_machineconfigpool=True)

Configure custom SSL certificate for ingress. If the certificate doesn’t exists, generate new one signed by automatic certificate signing service.

Parameters:

• skip_tls_verify (bool) – True if allow skipping TLS verification

• wait_for_machineconfigpool (bool) – True if it should wait for machineConfigPool

Raises:

ConfigurationError – when some required parameter is not configured

ocs_ci.utility.ssl_certs.configure_ingress_and_api_certificates(skip_tls_verify=False)

Configure custom SSL certificate for ingress and API.

Parameters:

skip_tls_verify (bool) – True if allow skipping TLS verification

ocs_ci.utility.ssl_certs.configure_trusted_ca_bundle(ca_cert_path, skip_tls_verify=False)

Configure cluster-wide trusted CA bundle in Proxy object

Parameters:

• ca_cert_path (str) – path to CA Certificate(s) bundle file

• skip_tls_verify (bool) – True if allow skipping TLS verification

ocs_ci.utility.ssl_certs.get_root_ca_cert()

If not available, download Root CA Certificate for custom Ingress certificate.

Returns

str: Path to Root CA Certificate

ocs_ci.utility.ssl_certs.init_arg_parser()

Init argument parser.

Returns:

Parsed arguments

Return type:

object

ocs_ci.utility.ssl_certs.main()

Main function for get-ssl-cert command

ocs_ci.utility.svt module

ocs_ci.utility.svt.svt_cleanup()

Removes clonned SVT project and virtual environemt and Projects Created while running SVT

Raises:

BaseException – In case any erros occured while removing project and ENV.

Returns:

True if No exceptions, False otherwise

Return type:

bool

ocs_ci.utility.svt.svt_cluster_loader(clusterload_file='/tmp/svt/openshift_scalability/config/master-vert.yaml')

ocs_ci.utility.svt.svt_create_venv_setup()

This function creates Virtual environemt for SVT project, and installs all the dependencies and activate the environment

ocs_ci.utility.svt.svt_project_clone()

This function clones the SVT project.

ocs_ci.utility.svt.svt_setup(iterations)

Creates and run svt workload

ocs_ci.utility.system module

ocs_ci.utility.system.is_path_empty(path)

Returns True if the given path does not contain any files, False otherwise.

Parameters:

path (str) – Path to be checked

ocs_ci.utility.templating module

class ocs_ci.utility.templating.Templating(base_path='/home/docs/checkouts/readthedocs.org/user_builds/ocs-ci/checkouts/latest/ocs_ci/templates')

Bases: object

Class which provides all functionality for templating

property base_path

Setter for self._base_path property

render_template(template_path, data)

Render a template with the given data.

Parameters:

• template_path (str) – location of the j2 template from the self._base_path

• data (dict) – the data to be formatted into the template

Returns: rendered template

ocs_ci.utility.templating.dump_data_to_json(data, json_file)

Dump data to json file

Parameters:

• data (dict) – dictionary with data to dump to the json file.

• json_file (str) – file path to json file

ocs_ci.utility.templating.dump_data_to_temp_yaml(data, temp_yaml)

Dump data to temporary yaml file

Parameters:

• data (dict or list) – dict or list (in case of multi_document) with data to dump to the yaml file.

• temp_yaml (str) – file path of yaml file

Returns:

dumped yaml data

Return type:

str

ocs_ci.utility.templating.dump_to_temp_yaml(src_file, dst_file, **kwargs)

Dump a jinja2 template file content into a yaml file

Args:

src_file (str): Template Yaml file path dst_file: the path to the destination Yaml file

ocs_ci.utility.templating.generate_yaml_from_jinja2_template_with_data(file_, **kwargs)

Generate yaml fron jinja2 yaml with processed data

Parameters:

file (str) – Template Yaml file path

Keyword Arguments:

attributes (All jinja2) –

Returns:

Generated from template file

Return type:

dict

Examples

generate_yaml_from_template(file_=’path/to/file/name’, pv_data_dict’)

ocs_ci.utility.templating.get_n_document_from_yaml(yaml_generator, index=0)

Returns n document from yaml generator loaded by load_yaml with multi_document = True.

Parameters:

• yaml_generator (generator) – Generator from yaml.safe_load_all

• index (int) – Index of document to return. (0 - 1st, 1 - 2nd document)

Returns:

Data from n document from yaml file.

Return type:

dict

Raises:

IndexError – In case that yaml generator doesn’t have such index.

ocs_ci.utility.templating.json_to_dict(json_file)

Converts JSON to dictionary format

Parameters:

json_file (str) – file path to json file

Returns:

JSON data in dictionary format

Return type:

dict

ocs_ci.utility.templating.load_config_data(data_path)

Loads YAML data from the specified path

Parameters:

data_path – location of the YAML data file

Returns: loaded YAML data

ocs_ci.utility.templating.load_yaml(file, multi_document=False)

Load yaml file (local or from URL) and convert it to dictionary

Parameters:

• file (str) – Path to the file or URL address

• multi_document (bool) – True if yaml contains more documents

Returns:

If multi_document == False, returns loaded data from yaml file

with one document.

generator: If multi_document == True, returns generator which each

iteration returns dict from one loaded document from a file.

Return type:

dict

ocs_ci.utility.templating.to_nice_yaml(a, indent=2, *args, **kw)

This is a j2 filter which allows you from dictionary to print nice human readable yaml.

Parameters:

• a (dict) – dictionary with data to print as yaml

• indent (int) – number of spaces for indent to be applied for whole dumped yaml. First line is not indented! (default: 2)

• *args – Other positional arguments which will be passed to yaml.dump

• *args – Other keywords arguments which will be passed to yaml.dump

Returns:

transformed yaml data in string format

Return type:

str

ocs_ci.utility.uninstall_openshift_logging module

Function to teardown the openshift-logging

ocs_ci.utility.uninstall_openshift_logging.check_pod_vanished(pod_names)

A function to check all the pods are vanished from the namespace

ocs_ci.utility.uninstall_openshift_logging.delete_logging_namespaces(force=False)

Deleting namespaces 1. Openshift-operators-redhat 2. Openshift-logging

ocs_ci.utility.uninstall_openshift_logging.uninstall_cluster_logging()

Function to uninstall cluster-logging from the cluster Deletes the project “openshift-logging” and “openshift-operators-redhat”

ocs_ci.utility.users module

ocs_ci.utility.users.add_htpasswd_user(username, password, htpasswd_path)

Create a new user credentials with provided username and password. These will be saved in file located on htpasswd_path. The file will be created if it doesn’t exist.

Parameters:

• username (str) – Name of a new user

• password (str) – Password for a new user

• htpasswd_path (str) – Path to httpasswd file

ocs_ci.utility.users.create_htpasswd_idp()

Create OAuth identity provider of HTPasswd type. It uses htpass-secret secret as a source for list of users.

ocs_ci.utility.users.create_htpasswd_secret(htpasswd_path, replace=False)

Create or update htpass-secret secret from file located on htpasswd_path.

Parameters:

• htpasswd_path (str) – Path to httpasswd file

• replace (bool) – If secret already exists then this will replace it

ocs_ci.utility.users.delete_htpasswd_secret()

Delete HTPasswd secret.

ocs_ci.utility.users.user_factory(request, htpasswd_path)

Create a user factory.

Parameters:

• request (obj) – request fixture

• htpasswd_path (str) – Path to htpasswd file

Returns:

User factory function

Return type:

func

ocs_ci.utility.utils module

class ocs_ci.utility.utils.AZInfo

Bases: object

A class for getting different az numbers across calls

get_zone_number()

Increment current zone_number and perform modulus op to roll-on to next available number

Returns:

zone number index

Return type:

int

zone_number = 0

class ocs_ci.utility.utils.TimeoutIterator(timeout, sleep, func, func_args=None, func_kwargs=None)

Bases: TimeoutSampler

Wrapper of TimeoutSampler which separates parameters of the class itself and func arguments in __init__ method. Such way of passing function with parameters is used in python standard library.

This allows more explicit usage, which improves readability, eg.:

t1 = TimeoutIterator(timeout=60, sleep=5, func=foo, func_args=[bar])
t2 = TimeoutIterator(3600, sleep=10, func=foo, func_args=[bar])

class ocs_ci.utility.utils.TimeoutSampler(timeout, sleep, func, *func_args, **func_kwargs)

Bases: object

Samples the function output.

This is a generator object that at first yields the output of function func. After the yield, it either raises instance of timeout_exc_cls or sleeps sleep seconds.

Yielding the output allows you to handle every value as you wish.

Feel free to set the instance variables.

Parameters:

• timeout (int) – Timeout in seconds

• sleep (int) – Sleep interval in seconds

• func (function) – The function to sample

• func_args – Arguments for the function

• func_kwargs – Keyword arguments for the function

wait_for_func_status(result)

Get function and run it for given time until success or timeout. (using __iter__ function)

Parameters:

result (bool) – Expected result from func.

Examples:

sample = TimeoutSampler(
    timeout=60, sleep=1, func=some_func, func_arg1="1",
    func_arg2="2"
)
if not sample.wait_for_func_status(result=True):
    raise Exception

wait_for_func_value(value)

Implements common usecase of TimeoutSampler: waiting until func (given function) returns a given value.

Parameters:

value – Expected return value of func we are waiting for.

ocs_ci.utility.utils.activate_multiple_mdss(mds_nodes)

ocs_ci.utility.utils.add_chrony_to_ocp_deployment()

Create and Add necessary chrony resources

ocs_ci.utility.utils.add_mem_stats(soup)

Add performance summary to the soup to print the table: columns = [‘TC name’, ‘Peak total RAM consumed’, ‘Peak total VMS consumed’, ‘RAM leak’]

ocs_ci.utility.utils.add_path_to_env_path(path)

Add path to the PATH environment variable (if not already there).

Parameters:

path (str) – Path which should be added to the PATH env. variable

ocs_ci.utility.utils.add_squad_analysis_to_email(session, soup)

Add squad analysis to the html test results used in email reporting

Parameters:

• session (obj) – Pytest session object

• soup (obj) – BeautifulSoup object of HTML Report data

ocs_ci.utility.utils.add_stage_cert()

Deploy stage certificate to the cluster.

ocs_ci.utility.utils.add_time_report_to_email(session, soup)

Takes the time report dictionary and converts it into HTML table

ocs_ci.utility.utils.allow_dir_fragmentation(mds_nodes)

ocs_ci.utility.utils.archive_ceph_crashes(toolbox_pod)

Archive all Ceph crashes

Parameters:

toolbox_pod (obj) – Ceph toolbox pod object

ocs_ci.utility.utils.auth_list(clients, mon_node)

ocs_ci.utility.utils.bin_xml_escape(arg)

Visually escape invalid XML characters.

For example, transforms ‘helloworld’ into ‘hello#x07world#x08’

Parameters:

arg (object) –

Returns:

string with escaped invalid characters

Return type:

str

ocs_ci.utility.utils.censor_values(data_to_censor)

This function censor string and numeric values in dictionary based on keys that match pattern defined in config_keys_patterns_to_censor in constants. It is performed recursively for nested dictionaries.

Parameters:

data_to_censor (dict) – Data to censor.

Returns:

filtered data

Return type:

dict

ocs_ci.utility.utils.ceph_crash_info_display(toolbox_pod)

Displays ceph crash information

Parameters:

toolbox_pod (obj) – Ceph toolbox pod object

ocs_ci.utility.utils.ceph_health_check(namespace=None, tries=20, delay=30)

Parameters:

• namespace (str) – Namespace of OCS (default: config.ENV_DATA[‘cluster_namespace’])

• tries (int) – Number of retries

• delay (int) – Delay in seconds between retries

Returns:

ceph_health_check_base return value with default retries of 20,

delay of 30 seconds if default values are not changed via args.

Return type:

bool

ocs_ci.utility.utils.ceph_health_check_base(namespace=None)

Exec ceph health cmd on tools pod to determine health of cluster.

Parameters:

namespace (str) – Namespace of OCS (default: config.ENV_DATA[‘cluster_namespace’])

Raises:

• CephHealthException – If the ceph health returned is not HEALTH_OK

• CommandFailed – If the command to retrieve the tools pod name or the command to get ceph health returns a non-zero exit code

Returns:

True if HEALTH_OK

Return type:

boolean

ocs_ci.utility.utils.ceph_health_check_multi_storagecluster_external(tries=20, delay=30)

Check ceph health for multi-storagecluster external.

ocs_ci.utility.utils.ceph_health_multi_storagecluster_external_base()

Check ceph health for multi-storagecluster external implementation.

Returns:

True if cluster health is ok.

Return type:

bool

Raises:

CephHealthException – Incase ceph health is not ok.

ocs_ci.utility.utils.check_for_rhcos_images(url)

Check for rhcos images are present in given location

Parameters:

url (str) – rhcos_images url

Returns:

True if images present if not false

Return type:

(bool)

ocs_ci.utility.utils.check_if_executable_in_path(exec_name)

Checks whether an executable can be found in the $PATH

Parameters:

exec_name – Name of executable to look for

Returns:

Whether the executable was found

Return type:

Boolean

ocs_ci.utility.utils.check_timeout_reached(start_time, timeout, err_msg=None)

Check if timeout reached and if so raise the exception.

Parameters:

• start_time (time) – Star time of the operation.

• timeout (int) – Timeout in seconds.

• err_msg (str) – Error message for the exception.

Raises:

TimeoutException – In case the timeout reached.

ocs_ci.utility.utils.clone_notify()

Repository contains the source code of notify tool, which is a python3 based tool wrapped by a container used to configure Ceph Bucket Notifications

Returns:

Path location of the notify code

Return type:

notify_path (str)

ocs_ci.utility.utils.clone_repo(url, location, tmp_repo=False, branch='master', to_checkout=None, clone_type='shallow', force_checkout=False)

Clone a repository or checkout latest changes if it already exists at

specified location.

Parameters:

• url (str) – location of the repository to clone

• location (str) – path where the repository will be cloned to

• tmp_repo (bool) – temporary repo, means it will be copied to temp path, to ‘location’

• branch (str) – branch name to checkout

• to_checkout (str) – commit id or tag to checkout

• clone_type (str) – type of clone (shallow, blobless, treeless and normal) By default, shallow clone will be used. For normal clone use clone_type as “normal”.

• force_checkout (bool) – True for force checkout to branch. force checkout will ignore the unmerged entries.

Raises:

UnknownCloneTypeException – In case of incorrect clone_type is used

ocs_ci.utility.utils.config_to_string(config)

Convert ConfigParser object to string in INI format.

Parameters:

config (obj) – ConfigParser object

Returns:

Config in one string

Return type:

str

ocs_ci.utility.utils.configure_chrony_and_wait_for_machineconfig_status(node_type='worker', timeout=900)

Configure chrony on the nodes

Parameters:

• node_type (str) – The node type to configure chrony e.g: worker, master and all if we want to configure on all nodes

• timeout (int) – Time in seconds to wait

ocs_ci.utility.utils.convert_bytes_to_unit(bytes_to_convert)

Convert bytes to bigger units like Kb, Mb, Gb or Tb.

Parameters:

bytes_to_convert (str) – The bytes to convert.

Returns:

The converted bytes as biggest unit possible

Return type:

str

ocs_ci.utility.utils.convert_device_size(unformatted_size, units_to_covert_to, convert_size=1000)

Convert a string representing a size to an int according to the given units to convert to

Parameters:

• unformatted_size (str) – The size to convert (i.e, ‘1Gi’/’100Mi’) acceptable units are: “Ti”, “Gi”, “Mi”, “Ki”, “Bi”

• units_to_covert_to (str) – The units to convert the size to (i.e, TB/GB/MB) acceptable units are: “TB”, “GB”, “MB”, “KB”, “BY”

• convert_size (int) – set convert by 1000 or 1024

Returns:

The converted size

Return type:

int

ocs_ci.utility.utils.convert_hostnames_to_ips(hostnames)

Gets the IP’s from hostname with FQDN

Parameters:

hostnames (list) – List of host names with FQDN

Returns:

Host IP’s

Return type:

list

ocs_ci.utility.utils.convert_yaml2tfvars(yaml)

Converts yaml file to tfvars. It creates the tfvars with the same filename in the required format which is used for deployment.

Parameters:

yaml (str) – File path to yaml

Returns:

File path to tfvars

Return type:

str

ocs_ci.utility.utils.create_ceph_health_cmd(namespace)

Forms the ceph health command

Parameters:

namespace (str) – Namespace of OCS

Returns:

ceph health command

Return type:

str

ocs_ci.utility.utils.create_directory_path(path)

Creates directory if path doesn’t exists

ocs_ci.utility.utils.create_rhelpod(namespace, pod_name, rhel_version=8, timeout=300)

Creates the RHEL pod

Parameters:

• namespace (str) – Namespace to create RHEL pod

• pod_name (str) – Pod name

• rhel_version (int) – RHEL version to be used for ansible

• timeout (int) – wait time for RHEL pod to be in Running state

Returns:

Pod instance for RHEL

Return type:

pod

ocs_ci.utility.utils.create_stats_dir()

create directory for Test run performance stats

ocs_ci.utility.utils.custom_ceph_config(suite_config, custom_config, custom_config_file)

Combines and returns custom configuration overrides for ceph. Hierarchy is as follows:

custom_config > custom_config_file > suite_config

Parameters:

• suite_config – ceph_conf_overrides that currently exist in the test suite

• custom_config – custom config args provided by the cli (these all go to the global scope)

• custom_config_file – path to custom config yaml file provided by the cli

Returns

New value to be used for ceph_conf_overrides in test config

ocs_ci.utility.utils.decode(encoded_message)

Decodes the message in base64

Parameters:

encoded_message (str) – encoded message

Returns:

decoded message

Return type:

str

ocs_ci.utility.utils.decompose_html_attributes(soup, attributes)

Decomposes the given html attributes

Parameters:

• soup (obj) – BeautifulSoup object

• attributes (list) – attributes to decompose

Returns: None

ocs_ci.utility.utils.delete_dir(dir_name)

Deletes the directory

Parameters:

dir_name (str) – Directory path to delete

ocs_ci.utility.utils.delete_file(file_name)

Delete file_name

Parameters:

file_name (str) – Path to the file you want to delete

ocs_ci.utility.utils.destroy_cluster(installer, cluster_path, log_level='DEBUG')

Destroy OCP cluster specific

Parameters:

• installer (str) – The path to the installer binary

• cluster_path (str) – The path of the cluster

• log_level (str) – log level openshift-installer (default: DEBUG)

ocs_ci.utility.utils.disable_huge_pages()

Removes huge pages

ocs_ci.utility.utils.download_file(url, filename, **kwargs)

Download a file from a specified url

Parameters:

• url (str) – URL of the file to download

• filename (str) – Name of the file to write the download to

• kwargs (dict) – additional keyword arguments passed to requests.get(…)

ocs_ci.utility.utils.download_file_from_git_repo(git_repo_url, path_to_file_in_git, filename)

Download a file from a specified git repository

Parameters:

• git_repo_url (str) – The git repository url

• path_to_file_in_git (str) – Path to the file to download in git repository

• filename (str) – Name of the file to write the download to

ocs_ci.utility.utils.dump_config_to_file(file_path)

Dump the config to the yaml file with censored secret values.

Parameters:

file_path (str) – Path to file where to write the configuration.

ocs_ci.utility.utils.email_reports(session)

Email results of test run

ocs_ci.utility.utils.enable_huge_pages()

Applies huge pages

ocs_ci.utility.utils.encode(message)

Encodes the message in base64

Parameters:

message (str/list) – message to encode

Returns:

encoded message in base64

Return type:

str

ocs_ci.utility.utils.ensure_nightly_build_availability(build_url)

ocs_ci.utility.utils.exec_cmd(cmd, secrets=None, timeout=600, ignore_error=False, threading_lock=None, silent=False, use_shell=False, cluster_config=None, **kwargs)

Run an arbitrary command locally

If the command is grep and matching pattern is not found, then this function returns “command terminated with exit code 1” in stderr.

Parameters:

• cmd (str) – command to run

• secrets (list) – A list of secrets to be masked with asterisks This kwarg is popped in order to not interfere with subprocess.run(**kwargs)

• timeout (int) – Timeout for the command, defaults to 600 seconds.

• ignore_error (bool) – True if ignore non zero return code and do not raise the exception.

• threading_lock (threading.RLock) – threading.RLock object that is used for handling concurrent oc commands

• silent (bool) – If True will silent errors from the server, default false

• use_shell (bool) – If True will pass the cmd without splitting

• cluster_config (MultiClusterConfig) – In case of multicluster environment this object will be non-null

Raises:

CommandFailed – In case the command execution fails

Returns:

(CompletedProcess) A CompletedProcess object of the command that was executed CompletedProcess attributes: args: The list or str args passed to run(). returncode (str): The exit code of the process, negative for signals. stdout (str): The standard output (None if not captured). stderr (str): The standard error (None if not captured).

ocs_ci.utility.utils.exec_nb_db_query(query)

Send a psql query to the Noobaa DB

Example usage:

exec_nb_db_query(“SELECT data ->> ‘key’ FROM objectmds;”)

Parameters:

query (str) – The query to send

Returns:

The query result rows

Return type:

list of str

ocs_ci.utility.utils.expose_ocp_version(version)

This helper function exposes latest nightly version or GA version of OCP. When the version string ends with .nightly (e.g. 4.2.0-0.nightly) it will expose the version to latest accepted OCP build (e.g. 4.2.0-0.nightly-2019-08-08-103722) If the version ends with -ga than it will find the latest GA OCP version and will expose 4.2-ga to for example 4.2.22.

Parameters:

version (str) – Verison of OCP

Returns:

Version of OCP exposed to full version if latest nighly passed

Return type:

str

ocs_ci.utility.utils.extract_ocp_binary_from_image(binary, image, bin_dir)

Extract binary (oc client or openshift installer) from custom OCP image

Parameters:

• binary (str) – type of binary (oc or openshift-install)

• image (str) – image URL

• bin_dir (str) – path to bin folder where to extract the binary

ocs_ci.utility.utils.file_locking(client)

ocs_ci.utility.utils.filter_out_emojis(plaintext)

Filter out emojis from a string

Parameters:

string (str) – String to filter out emojis from

Returns:

Filtered string

Return type:

str

ocs_ci.utility.utils.filter_unrepresentable_values(data_to_filter)

This function filter values in dictionary or list which are not possible convert to yaml (e.g. objects), to prevent following error raised from yaml.safe_dump yaml.representer.RepresenterError(“cannot represent an object”,…) It is performed recursively for nested dictionaries or lists.

Parameters:

data_to_filter (dict|list|tuple) – Data to censor.

Returns:

filtered data

Return type:

dict

ocs_ci.utility.utils.fuse_client_io(client, mounting_dir)

ocs_ci.utility.utils.fuse_client_md5(fuse_clients, md5sum_list1)

ocs_ci.utility.utils.fuse_mount(fuse_clients, mounting_dir)

ocs_ci.utility.utils.get_attr_chain(obj, attr_chain)

Attempt to retrieve object attributes when uncertain about the existence of the attribute or a different attribute in a given attribute chain. If the retrieval fails, None is returned. The function can be used to retrieve a direct attribute, or a chain of attributes. i.e. - obj.attr_a, obj_attr_a.sub_attr

Another example - trying to access “sub_attr_b” in object.attr.sub_attr_a.sub_attr_b - get_attr_chain(object, “attr.sub_attr_a.sub_attr_b”)

The function can be used to try and retrieve “sub_attribute_b” without an exception, even in cases where “attr” or “sub_attr_a” might not exist. In those cases, the function will return None.

Parameters:

• obj – An object

• attr_chain (str) – A string containing one attribute or several sub-attributes separated by dots (i.e. - “attr.sub_attr_a.sub_attr_b”)

Returns:

The requested attribute if found, otherwise None

ocs_ci.utility.utils.get_available_ocp_versions(channel)

Find all available OCP versions for specific channel.

Parameters:

channel (str) – Channel of OCP (e.g. stable-4.2 or fast-4.2)

Returns

list: Sorted list with OCP versions for specified channel.

ocs_ci.utility.utils.get_az_count()

Using a number of different configuration attributes, determine how many availability zones the cluster is configured for.

Returns:

number of availability zones

Return type:

int

ocs_ci.utility.utils.get_ceph_crashes(toolbox_pod)

Gets all Ceph crashes

Parameters:

toolbox_pod (obj) – Ceph toolbox pod object

Returns:

List of ceph crash ID’s

Return type:

list

ocs_ci.utility.utils.get_ceph_version()

Gets the ceph version

Returns:

ceph version

Return type:

str

ocs_ci.utility.utils.get_client_info(ceph_nodes, clients)

ocs_ci.utility.utils.get_client_version(client_binary_path)

Get version reported by oc version.

Parameters:

client_binary_path (str) – path to oc binary

Returns:

version reported by oc version.

None if the client does not exist at the provided path.

Return type:

str

ocs_ci.utility.utils.get_cluster_id(cluster_path)

Get ClusterID from metadata.json in given cluster_path

Parameters:

cluster_path – path to cluster install directory

Returns:

metadata.json[‘clusterID’]

Return type:

str

ocs_ci.utility.utils.get_cluster_image()

Gets the cluster image

Returns:

cluster image

Return type:

str

ocs_ci.utility.utils.get_cluster_name(cluster_path)

Get clusterName from metadata.json in given cluster_path

Parameters:

cluster_path – path to cluster install directory

Returns:

metadata.json[‘clusterName’]

Return type:

str

ocs_ci.utility.utils.get_cluster_version()

Gets the cluster version

Returns:

cluster version

Return type:

str

ocs_ci.utility.utils.get_cluster_version_info()

Gets the complete cluster version information

Returns:

cluster version information

Return type:

dict

ocs_ci.utility.utils.get_csi_versions()

Gets the CSI related version information

Returns:

CSI related version information

Return type:

dict

ocs_ci.utility.utils.get_default_if_keyval_empty(dictionary, key, default_val)

if Key has an empty value OR key doesn’t exist then return default value

Parameters:

• dictionary (dict) – Dictionary where we have to lookup

• key (str) – key to lookup

• default_val (str) – If key doesn’t have value then return this default_val

Returns:

dictionary[key] if value is present else default_val

ocs_ci.utility.utils.get_image_with_digest(image)

Return image with sha256 digest for usage in disconnected environment

Parameters:

image (str) – image

Raises:

UnexpectedImage – In case the image information is unexpected

Returns:

image with sha256 digest specification

Return type:

str

ocs_ci.utility.utils.get_infra_id(cluster_path)

Get infraID from metadata.json in given cluster_path

Parameters:

cluster_path – path to cluster install directory

Returns:

metadata.json[‘infraID’]

Return type:

str

ocs_ci.utility.utils.get_infra_id_from_openshift_install_state(cluster_path)

Get infraID from openshift_install_state.json in given cluster_path

Parameters:

cluster_path – path to cluster install directory

Returns:

cluster infraID

Return type:

str

ocs_ci.utility.utils.get_kubeadmin_password()

ocs_ci.utility.utils.get_latest_acm_tag_unreleased(version)

Get Latest tag for acm unreleased image

Args:

version (str): version of acm for getting latest tag

Returns:

image tag for the specified version

Return type:

str

Raises:

TagNotFoundException – When the given version is not found

ocs_ci.utility.utils.get_latest_ds_olm_tag(upgrade=False, latest_tag=None)

This function returns latest tag of OCS downstream registry or one before latest if upgrade parameter is True

Parameters:

• upgrade (str) – If True then it returns one version of the build before the latest.

• latest_tag (str) – Tag of the latest build. If not specified config.DEPLOYMENT[‘default_latest_tag’] or ‘latest’ will be used.

Returns:

latest tag for downstream image from quay registry

Return type:

str

Raises:

TagNotFoundException – In case no tag found

ocs_ci.utility.utils.get_latest_ocp_version(channel, index=-1)

Find latest OCP version for specific channel.

Parameters:

• channel (str) – Channel of OCP (e.g. stable-4.2 or fast-4.2)

• index (int) – Index to get from all available versions list e.g. default -1 is latest version (version[-1]). If you want to get previous version pass index -2 and so on.

Returns

str: Latest OCP version for specified channel.

ocs_ci.utility.utils.get_module_ip(terraform_state_file, module)

Gets the node IP from terraform.tfstate file

Parameters:

• terraform_state_file (str) – Path to terraform state file

• module (str) – Module name in terraform.tfstate file e.g: constants.LOAD_BALANCER_MODULE

Returns:

IP of the node

Return type:

list

ocs_ci.utility.utils.get_next_version_available_for_upgrade(current_tag)

This function returns the tag built after the current_version

Parameters:

current_tag (str) – Current build tag from which to search the next one build tag.

Returns:

tag for downstream image from quay registry built after

the current_tag.

Return type:

str

Raises:

TagNotFoundException – In case no tag suitable for upgrade found

ocs_ci.utility.utils.get_oadp_version()

Returns:

returns version string

Return type:

str

ocs_ci.utility.utils.get_ocm_cli(version=None, bin_dir=None, force_download=False)

Download the OCM binary, if not already present. Update env. PATH and get path of the OCM binary.

Parameters:

• version (str) – Version of the OCM to download

• bin_dir (str) – Path to bin directory (default: config.RUN[‘bin_dir’])

• force_download (bool) – Force OCM download even if already present

Returns:

Path to the OCM binary

Return type:

str

ocs_ci.utility.utils.get_ocp_repo(rhel_major_version=None)

Get ocp repo file, name will be generated dynamically based on ocp version.

Parameters:

rhel_major_version (int) – Major version of RHEL. If not specified it will take major version from config.ENV_DATA[“rhel_version”]

Returns:

Path to ocp repo file

Return type:

string

ocs_ci.utility.utils.get_ocp_upgrade_history()

Gets the OCP upgrade history for the cluster

Returns:

List of OCP upgrade paths. Latest version in the

beginning of the list

Return type:

list

ocs_ci.utility.utils.get_ocp_version(seperator=None)

The deprecated form of ‘get current ocp version’ Use ocs_ci/utility/version.py:get_semantic_ocp_version_from_config()

Get current ocp version

Parameters:

seperator (str) – String that would seperate major and minor version nubers

Returns:

If seperator is ‘None’, version string will be returned as is

eg: ‘4.2’, ‘4.3’. If seperator is provided then ‘.’ in the version string would be replaced by seperator and resulting string will be returned. eg: If seperator is ‘_’ then string returned would be ‘4_2’

Return type:

string

ocs_ci.utility.utils.get_ocs_build_number()

Gets the build number for ocs operator

Returns:

build number for ocs operator version

Return type:

str

ocs_ci.utility.utils.get_ocs_olm_operator_tags(limit=100)

Query the OCS OLM Operator repo and retrieve a list of tags. Since we are limited to 100 tags per page, we end up making several API calls and combining the results into a single list of tags.

Parameters:

limit – the number of tags to limit the request to

Raises:

• KeyError – if the auth config isn’t setup properly

• requests.RequestException – if the response return code is not ok

Returns:

OCS OLM Operator tags

Return type:

list

ocs_ci.utility.utils.get_ocs_version_from_image(image)

Parse major.minor version from OCS image tag.

Parameters:

image (str) – image in format url:tag

Returns

str: Version in x.y format

Raises:

ValueError – In case of the tag which we cannot parse to version.

ocs_ci.utility.utils.get_openshift_client(version=None, bin_dir=None, force_download=False, skip_comparison=False)

Download the OpenShift client binary, if not already present. Update env. PATH and get path of the oc binary.

Parameters:

• version (str) – Version of the client to download (default: config.RUN[‘client_version’])

• bin_dir (str) – Path to bin directory (default: config.RUN[‘bin_dir’])

• force_download (bool) – Force client download even if already present

• skip_comparison (bool) – Skip the comparison between the existing OCP client version and the configured one.

Returns:

Path to the client binary

Return type:

str

ocs_ci.utility.utils.get_openshift_installer(version=None, bin_dir=None, force_download=False)

Download the OpenShift installer binary, if not already present. Update env. PATH and get path of the openshift installer binary.

Parameters:

• version (str) – Version of the installer to download

• bin_dir (str) – Path to bin directory (default: config.RUN[‘bin_dir’])

• force_download (bool) – Force installer download even if already present

Returns:

Path to the installer binary

Return type:

str

ocs_ci.utility.utils.get_openshift_mirror_url(file_name, version)

Format url to OpenShift mirror (for client and installer download).

Parameters:

• file_name (str) – Name of file

• version (str) – Version of the installer or client to download

Returns:

Url of the desired file (installer or client)

Return type:

str

Raises:

• UnsupportedOSType – In case the OS type is not supported

• UnavailableBuildException – In case the build url is not reachable

ocs_ci.utility.utils.get_pytest_fixture_value(request, fixture_name)

Get the value of a fixture name from the request

Parameters:

• request (_pytest.fixtures.SubRequest') – The pytest request fixture

• fixture_name – Fixture for which this request is being performed

Returns:

The fixture value

Return type:

Any

ocs_ci.utility.utils.get_random_str(size=13)

generates the random string of given size

Parameters:

size (int) – number of random characters to generate

Returns:

string of random characters of given size

Return type:

str

ocs_ci.utility.utils.get_role_arn_from_sub()

Get the RoleARN from the OCS subscription

Returns:

Role ARN used for ODF deployment

Return type:

role_arn (str)

Raises:

• ClusterNotInSTSModeException (Exception) if cluster –

• not in STS mode –

ocs_ci.utility.utils.get_rook_repo(branch='master', to_checkout=None)

Clone and checkout the rook repository to specific branch/commit.

Parameters:

• branch (str) – Branch name to checkout

• to_checkout (str) – Commit id or tag to checkout

ocs_ci.utility.utils.get_rook_version()

Gets the rook version

Returns:

rook version

Return type:

str

ocs_ci.utility.utils.get_root_disk(node)

Fetches the root (boot) disk for node

Parameters:

node (str) – Node name

Returns:

Root disk

Return type:

str

ocs_ci.utility.utils.get_rosa_cli(version=None, bin_dir=None, force_download=False)

Download the ROSA binary, if not already present. Update env. PATH and get path of the ROSA binary.

Parameters:

• version (str) – Version of the ROSA to download

• bin_dir (str) – Path to bin directory (default: config.RUN[‘bin_dir’])

• force_download (bool) – Force ROSA download even if already present

Returns:

Path to the rosa binary

Return type:

str

ocs_ci.utility.utils.get_running_acm_version()

Get the currently deployed ACM version

Returns:

ACM version

Return type:

string

ocs_ci.utility.utils.get_running_cluster_id()

Get cluster UUID Not relying on metadata.json as user sometimes want to run only with kubeconfig for some tests. For this function to work cluster has to be in running state

Returns:

cluster UUID

Return type:

str

ocs_ci.utility.utils.get_running_ocp_version(separator=None)

Get current running ocp version

Parameters:

separator (str) – String that would separate major and minor version numbers

Returns:

If separator is ‘None’, version string will be returned as is

eg: ‘4.2’, ‘4.3’. If separator is provided then ‘.’ in the version string would be replaced by separator and resulting string will be returned. eg: If separator is ‘_’ then string returned would be ‘4_2’

Return type:

string

ocs_ci.utility.utils.get_system_architecture()

Get output from ‘uname -m’ command run on first worker node.

Returns:

Architecture of system

Return type:

str

ocs_ci.utility.utils.get_terraform(version=None, bin_dir=None)

Downloads the terraform binary

Parameters:

• version (str) – Version of the terraform to download

• bin_dir (str) – Path to bin directory (default: config.RUN[‘bin_dir’])

Returns:

Path to the terraform binary

Return type:

str

ocs_ci.utility.utils.get_terraform_ignition_provider(terraform_dir, version=None)

Downloads the terraform ignition provider

Parameters:

• terraform_dir (str) – Path to terraform working directory

• version (str) – Version of the terraform ignition provider to download

ocs_ci.utility.utils.get_testrun_name()

Prepare testrun ID for Polarion (and other reports).

Returns:

String containing testrun name

Return type:

str

ocs_ci.utility.utils.get_trim_mean(values, percentage=20)

Get the trimmed mean of a list of values. Explanation: This function finds the arithmetic mean of given values, ignoring values outside the given limits.

Parameters:

• values (list) – The list of values

• percentage (int) – The percentage to be trimmed

Returns:

Trimmed mean. In case trimmed mean calculation fails,

the regular mean average is returned

Return type:

float

ocs_ci.utility.utils.get_url_content(url, **kwargs)

Return URL content

Parameters:

• url (str) – URL address to return

• kwargs (dict) – additional keyword arguments passed to requests.get(…)

Returns:

Content of URL

Return type:

str

Raises:

AssertionError – When couldn’t load URL

ocs_ci.utility.utils.get_vault_cli(bind_dir=None, force_download=False)

Download vault based on platform basically for CLI purpose. Binary will be directly put into ocs_ci/bin/ directory

Parameters:

• bind_dir (str) – Path to bin directory (default: config.RUN[‘bin_dir’])

• force_download (bool) – Force vault cli download even if already present

ocs_ci.utility.utils.inspect_image(image, authfile_fo, cluster_config=None)

Inspect image

Parameters:

• image (str) – image to inspect

• authfile_fo (NamedTemporaryFile) – pull-secret required for pulling the given image

• cluster_config (MultiClusterConfig) – Holds the context of a cluster

Returns:

json object of the inspected image

Return type:

dict

ocs_ci.utility.utils.is_cluster_running(cluster_path)

ocs_ci.utility.utils.is_cluster_y_version_upgraded()

Checks whether cluster is upgraded or not

Returns:

True if cluster is upgraded from previous versions

Return type:

bool

ocs_ci.utility.utils.is_emoji(char)

ocs_ci.utility.utils.kernel_client_io(client, mounting_dir)

ocs_ci.utility.utils.kernel_client_md5(kernel_clients, md5sum_list2)

ocs_ci.utility.utils.kernel_mount(mounting_dir, mon_node_ip, kernel_clients)

ocs_ci.utility.utils.list_insert_at_position(lst, index, element)

Insert an element into the list at a specific index while shifting all the element one setp right to the index

ocs_ci.utility.utils.load_auth_config()

Load the authentication config YAML from /data/auth.yaml

Raises:

FileNotFoundError – if the auth config is not found

Returns:

A dictionary reprensenting the YAML file

Return type:

dict

ocs_ci.utility.utils.load_config_file(config_file)

Loads config file to the ocs-ci config

Parameters:

config_file (str) – Path to yaml config file.

Raises:

FileNotFoundError – In the case the config file not found.

ocs_ci.utility.utils.login_to_mirror_registry(authfile, cluster_config=None)

Login to mirror registry

Parameters:

authfile (str) – authfile (pull-secret) path

ocs_ci.utility.utils.mask_secrets(plaintext, secrets)

Replace secrets in plaintext with asterisks

Parameters:

• plaintext (str or list) – The plaintext to remove the secrets from or list of strings to remove secrets from

• secrets (list) – List of secret strings to replace in the plaintext

Returns:

The censored version of plaintext

Return type:

str

ocs_ci.utility.utils.mds_fail_over(mds_nodes)

ocs_ci.utility.utils.mirror_image(image, cluster_config=None)

Mirror image to mirror image registry.

Parameters:

• image (str) – image to be mirrored, can be defined just with name or with full url, with or without tag or digest

• cluster_config (MultiClusterConfig) – Config object if single cluster, if its multicluster scenario then we will have MultiClusterConfig object

Returns:

the mirrored image link

Return type:

str

ocs_ci.utility.utils.mkdir_pinning(clients, range1, range2, dir_name, pin_val)

ocs_ci.utility.utils.modify_csv(csv, replace_from, replace_to)

Modify the CSV

Parameters:

• csv (str) – The CSV name

• replace_from (str) – The pattern to replace from in the CSV

• replace_to (str) – The pattern to replace to in the CSV

ocs_ci.utility.utils.move_summary_to_top(soup)

Move summary to the top of the eamil report

ocs_ci.utility.utils.ocsci_log_path()

Construct the full path for the log directory.

Returns:

full path for ocs-ci log directory

Return type:

str

ocs_ci.utility.utils.parse_html_for_email(soup)

Parses the html and filters out the unnecessary data/tags/attributes for email reporting

Parameters:

soup (obj) – BeautifulSoup object

ocs_ci.utility.utils.parse_pgsql_logs(data)

Parse the pgsql benchmark data from ripsaw and return the data in list format

Parameters:

data (str) – log data from pgsql bench run

Returns:

data digestable by scripts with below format

e.g.:

[ {1: {‘num_clients’: ‘2’,’num_threads’: ‘7’,’latency_avg’: ‘7’, ‘lat_stddev’: ‘0’, ‘tps_incl’: ‘234’, ‘tps_excl’: ‘243’}, {2: {‘num_clients’: ‘2’,’num_threads’: ‘7’,’latency_avg’: ‘7’, ‘lat_stddev’: ‘0’, ‘tps_incl’: ‘234’, ‘tps_excl’: ‘243’}, {3: {‘num_clients’: ‘2’,’num_threads’: ‘7’,’latency_avg’: ‘7’, ‘lat_stddev’: ‘0’, ‘tps_incl’: ‘234’, ‘tps_excl’: ‘243’}, ] where keys{1,2,3} are run-IDs

Return type:

list_data (list)

ocs_ci.utility.utils.pinned_dir_io(clients, mds_fail_over, num_of_files, range1, range2)

ocs_ci.utility.utils.prepare_bin_dir(bin_dir=None)

Prepare bin directory for OpenShift client and installer

Parameters:

bin_dir (str) – Path to bin directory (default: config.RUN[‘bin_dir’])

ocs_ci.utility.utils.prepare_customized_pull_secret(images=None)

Prepare customized pull-secret containing auth section related to given image(s). If image(s) not defined or no related section is found, it will use whole content of pull-secret.

Parameters:

images (str, list) – image (or images) to match with auth section

Returns:

prepared pull-secret

Return type:

NamedTemporaryFile

ocs_ci.utility.utils.read_file_as_str(filepath)

Reads the file content

Parameters:

filepath (str) – File to read

Returns:

File contents in string

Return type:

str

ocs_ci.utility.utils.remove_ceph_crashes(toolbox_pod)

Deletes the Ceph crashes

Parameters:

toolbox_pod (obj) – Ceph toolbox pod object

ocs_ci.utility.utils.remove_keys_from_tf_variable_file(tf_file, keys)

Removes the keys from the tf files and convert to json format

Parameters:

• tf_file (str) – path to tf file

• keys (list) – list of keys to remove

ocs_ci.utility.utils.replace_content_in_file(file, old, new, match_and_replace_line=False)

Replaces contents in file, if old value is not found, it adds new value to the file

Parameters:

• file (str) – Name of the file in which contents will be replaced

• old (str) – Data to search for

• new (str) – Data to replace the old value

• match_and_replace_line (bool) – If True, it will match a line if old pattern is found in the line. The whole line will be replaced with new content. Otherwise it will replace only old string with new string but the rest of the line will be intact. This is the default option.

ocs_ci.utility.utils.run_async(command)

Run command locally and return without waiting for completion

Parameters:

command (str) – The command to run.

Returns:

An open descriptor to be used by the calling function.

Example

command = ‘oc delete pvc pvc1’ proc = run_async(command) ret, out, err = proc.async_communicate()

ocs_ci.utility.utils.run_ceph_health_cmd(namespace)

Run the ceph health command

Parameters:

namespace – Namespace of OCS

Raises:

CommandFailed – In case the rook-ceph-tools pod failed to reach the Ready state.

Returns:

The output of the ceph health command

Return type:

str

ocs_ci.utility.utils.run_cmd(cmd, secrets=None, timeout=600, ignore_error=False, threading_lock=None, silent=False, cluster_config=None, **kwargs)

The deprecated form of exec_cmd. Run an arbitrary command locally

Parameters:

• cmd (str) – command to run

• secrets (list) – A list of secrets to be masked with asterisks This kwarg is popped in order to not interfere with subprocess.run(**kwargs)

• timeout (int) – Timeout for the command, defaults to 600 seconds.

• ignore_error (bool) – True if ignore non zero return code and do not raise the exception.

• threading_lock (threading.RLock) – threading.RLock object that is used for handling concurrent oc commands

• silent (bool) – If True will silent errors from the server, default false

Raises:

CommandFailed – In case the command execution fails

Returns:

(str) Decoded stdout of command

ocs_ci.utility.utils.run_cmd_interactive(cmd, prompts_answers, timeout=300)

Handle interactive prompts with answers during subctl command

Parameters:

• cmd (str) – Command to be executed

• prompts_answers (dict) – Prompts as keys and answers as values

• timeout (int) – Timeout in seconds, for pexpect to wait for prompt

Raises:

InteractivePromptException – in case something goes wrong

ocs_ci.utility.utils.run_cmd_multicluster(cmd, secrets=None, timeout=600, ignore_error=False, skip_index=None, **kwargs)

Run command on multiple clusters. Useful in multicluster scenarios This is wrapper around exec_cmd

Parameters:

• cmd (str) – command to be run

• secrets (list) – A list of secrets to be masked with asterisks This kwarg is popped in order to not interfere with subprocess.run(**kwargs)

• timeout (int) – Timeout for the command, defaults to 600 seconds.

• ignore_error (bool) – True if ignore non zero return code and do not raise the exception.

• skip_index (list of int) – List of indexes that needs to be skipped from executing the command

Raises:

CommandFailed – In case the command execution fails

Returns:

of CompletedProcess objects as per cluster’s index in config.clusters

i.e. [cluster1_completedprocess, None, cluster2_completedprocess] if command execution skipped on a particular cluster then corresponding entry will have None

Return type:

list

ocs_ci.utility.utils.save_reports()

Save reports of test run to logs directory

ocs_ci.utility.utils.set_aws_region(region=None)

Exports environment variable AWS_REGION

Parameters:

region (str) – AWS region to export

ocs_ci.utility.utils.set_registry_to_managed_state()

In order to be able to deploy from stage we need to change image registry config to Managed state. More described in BZs: https://bugzilla.redhat.com/show_bug.cgi?id=1806593 https://bugzilla.redhat.com/show_bug.cgi?id=1807471#c3 We need to change to managed state as described here: https://github.com/red-hat-storage/ocs-ci/issues/1436 So this is not suppose to be deleted as WA case we really need to do this operation for OCS deployment as was originally done here: https://github.com/red-hat-storage/ocs-ci/pull/1437 Currently it has to be moved here to enable CA certificate to be properly propagated for the stage deployment as mentioned in BZ.

ocs_ci.utility.utils.set_selinux_permissions(workers=None)

Workaround for #1777384 - enable container_use_cephfs on RHEL workers Ticket: RHSTOR-787, see more details in the issue: #1151

Parameters:

workers (list) – List of worker nodes to set selinux permissions

ocs_ci.utility.utils.skipif_ocp_version(expressions)

This function evaluates the condition for test skip based on expression

Parameters:

• expressions (str OR list) – condition for which we need to check,

• eg – A single expression string ‘>=4.2’ OR A list of expressions like [‘<4.3’, ‘>4.2’], [‘<=4.3’, ‘>=4.2’]

Returns:

‘True’ if test needs to be skipped else ‘False’

ocs_ci.utility.utils.skipif_ocs_version(expressions)

This function evaluates the condition for test skip based on expression

Parameters:

• expressions (str OR list) – condition for which we need to check,

• eg – A single expression string ‘>=4.2’ OR A list of expressions like [‘<4.3’, ‘>4.2’], [‘<=4.3’, ‘>=4.2’]

Returns:

‘True’ if test needs to be skipped else ‘False’

ocs_ci.utility.utils.skipif_ui_not_support(ui_test)

This function evaluates the condition for ui test skip based on ui_test expression

Parameters:

ui_test (str) – condition for which we need to check,

Returns:

‘True’ if test needs to be skipped else ‘False’

ocs_ci.utility.utils.skipif_upgraded_from(version_list)

This function evaluates the condition to skip a test if the cluster is upgraded from a particular OCS version

Parameters:

version_list (list) – List of versions to check

Returns:

True if test needs to be skipped else False

Return type:

(bool)

ocs_ci.utility.utils.string_chunkify(cstring, csize)

Create string chunks of size csize from cstring and yield chunk by chunk

Parameters:

• cstring (str) – Original string which need to be chunkified

• csize (int) – size of each chunk

ocs_ci.utility.utils.switch_to_correct_cluster_at_setup(request)

Switch to the correct cluster index at setup, according to the ‘cluster_type’ fixture parameter provided in the test.

Parameters:

request (_pytest.fixtures.SubRequest') – The pytest request fixture

ocs_ci.utility.utils.update_container_with_mirrored_image(job_pod_dict)

Update Job or Pod configuration dict with mirrored image (required for disconnected installation).

Parameters:

job_pod_dict (dict) – dictionary with Job or Pod configuration

Returns:

for disconnected installation, returns updated Job or Pod dict,

for normal installation return unchanged job_pod_dict

Return type:

dict

ocs_ci.utility.utils.upload_file(server, localpath, remotepath, user=None, password=None, key_file=None)

Upload a file to remote server

Parameters:

• server (str) – Name of the server to upload

• localpath (str) – Local file to upload

• remotepath (str) – Target path on the remote server. filename should be included

• user (str) – User to use for the remote connection

ocs_ci.utility.utils.wait_for_ceph_health_not_ok(timeout=300, sleep=10)

Wait until the ceph health is NOT OK

ocs_ci.utility.utils.wait_for_co(operator)

Waits for ClusterOperator to created

Parameters:

operator (str) – Name of the ClusterOperator

ocs_ci.utility.utils.wait_for_machineconfigpool_status(node_type, timeout=1900, skip_tls_verify=False)

Check for Machineconfigpool status

Parameters:

• node_type (str) – The node type to check machineconfigpool status is updated. e.g: worker, master and all if we want to check for all nodes

• timeout (int) – Time in seconds to wait

• skip_tls_verify (bool) – True if allow skipping TLS verification

ocs_ci.utility.utils.wipe_all_disk_partitions_for_node(node)

Wipes out partition for all disks which has “nvme” prefix

Parameters:

node (str) – Name of the node (OCP Node)

ocs_ci.utility.utils.wipe_partition(node, disk_path)

Wipes out partition for disk using sgdisk

Parameters:

• node (str) – Name of the node (OCP Node)

• disk_path (str) – Disk to wipe partition

ocs_ci.utility.version module

Module for version related util functions.

ocs_ci.utility.version.compare_versions(expression)

Evaluate version comparison expression

Parameters:

expression (str) – expression string like ‘4.11>=4.2’, supported operators are: >,<,=>,=<,==,!=

Returns:

evaluated comparison expression

Return type:

Boolean

ocs_ci.utility.version.get_ocs_version_from_csv(only_major_minor=False, ignore_pre_release=False)

Returns semantic OCS Version from the CSV (ODF if version >= 4.9, OCS otherwise)

Parameters:

• only_major_minor (bool) – If True, only major and minor will be parsed.

• ignore_pre_release (bool) – If True, the pre release version will be ignored

Returns:

Object of semantic version for OCS.

Return type:

semantic_version.base.Version

ocs_ci.utility.version.get_semantic_ocp_running_version(separator=None)

Returning running OCP semantic version from cluster.

Parameters:

separator (str) – String that would separate major and minor version numbers

Returns:

Object of semantic version for OCP.

Return type:

semantic_version.base.Version

ocs_ci.utility.version.get_semantic_ocp_version_from_config()

Returning OCP semantic version from config.

Returns:

Object of semantic version for OCP.

Return type:

semantic_version.base.Version

ocs_ci.utility.version.get_semantic_ocs_version_from_config(cluster_config=None)

Returning OCS semantic version from config.

Returns:

Object of semantic version for OCS. cluster_config (MultiClusterConfig): config related to specific cluster in case of multicluster

Return type:

semantic_version.base.Version

ocs_ci.utility.version.get_semantic_version(version, only_major_minor=False, ignore_pre_release=False)

Returning semantic version from provided version as string.

Parameters:

• version (str) – String version (e.g. 4.6)

• only_major_minor (bool) – If True, only major and minor will be parsed.

• ignore_pre_release (bool) – If True, the pre release version will be ignored

Returns:

Object of semantic version.

Return type:

semantic_version.base.Version

ocs_ci.utility.vsphere module

This module contains the vSphere related methods

class ocs_ci.utility.vsphere.VSPHERE(host, user, password, port=443)

Bases: object

wrapper for vSphere

add_disk(vm, size, disk_type='thin')

Attaches disk to VM

Parameters:

• vm (vim.VirtualMachine) – VM instance

• size (int) – size of disk in GB

• disk_type (str) – disk type

add_disks(num_disks, vm, size, disk_type='thin')

Adds multiple disks to the VM

Parameters:

• num_disks – number of disks to add

• vm (vim.VirtualMachine) – VM instance

• size (int) – size of disk in GB

• disk_type (str) – disk type

add_pci_device(vm, pci_device)

Attaches PCI device to VM

Parameters:

• vm (vim.VirtualMachine) – VM instance

• pci_device (vim.vm.PciPassthroughInfo) – PCI device to add

add_rdm_disk(vm, device_name, disk_mode=None, compatibility_mode=None)

Attaches RDM disk to vm

Parameters:

• vm (vim.VirtualMachine) – VM instance

• device_name (str) – Device name to add to VM. e.g:”/vmfs/devices/disks/naa.600304801b540c0125ef160f3048faba”

• disk_mode (str) – Disk mode. By default it will add ‘independent_persistent’. Available modes are ‘append’, ‘independent_nonpersistent’, ‘independent_persistent’, ‘nonpersistent’, ‘persistent’, and ‘undoable’

• compatibility_mode (str) – Compatabilty mode. Either ‘physicalMode’ or ‘virtualMode’. By default it will add ‘physicalMode’.

available_storage_devices(host, datastore_type='VMFS')

Fetches the available storage devices on Host.

Parameters:

• host (vim.HostSystem) – Host instance

• datastore_type (str) – Type of datastore. Either VMFS or vsan By default, it will take VMFS as datastore type.

Returns:

List of storage devices available for use

Return type:

list

change_vm_network_state(ip, dc, label='Network adapter 1', network='VM Network', timeout=10, connect=True)

Connects or disconnects a VM’s network adapter.

Parameters:

• ip (str) – The IP address of the VM.

• dc (str) – The datacenter name where the VM is located.

• label (str, optional) – The label of the network adapter to disconnect. Defaults to “Network adapter 1”.

• network (str, optional) – The name of the network to disconnect from. Defaults to “VM Network”.

• timeout (int, optional) – The maximum time to wait for the operation to complete. Defaults to 10 seconds.

• connect (bool, optional) – True to connect the network adapter, False to disconnect. Defaults to True.

Returns:

True if the operation was successful, False otherwise.

Return type:

bool

check_folder_exists(name, cluster, dc)

Checks whether folder exists in Templates

Parameters:

• name (str) – Folder name

• cluster (str) – Cluster name

• dc (str) – Datacenter name

Returns:

True if folder exists, False otherwise

Return type:

bool

clone_vm(vm_name, template_name, datacenter_name, resource_pool_name, datastore_name, cluster_name, cpus=4, memory=8, root_disk_size=125829120, network_adapter='VM Network', power_on=True, **kwargs)

Clones the VM from template

Parameters:

• vm_name (str) – Name of the VM to create

• template_name (str) – Template name to clone

• datacenter_name (str) – Name of the Datacenter

• resource_pool_name (str) – Name of the Resource Pool

• datastore_name (str) – Name of the Datastore

• cluster_name (str) – Name of the Cluster in Datacenter

• cpus (int) – Number of CPU’s

• memory (int) – Memory in MB

• root_disk_size (int) – Root Disk size in KB

• network_adapter (str) – Name of the Network Adapter

• power_on (bool) – True to power on the VM after cloning

destroy_folder(name, cluster, dc)

Removes the folder from Templates

Parameters:

• name (str) – Folder name

• cluster (str) – Cluster name

• dc (str) – Datacenter name

destroy_pool(pool, dc, cluster)

Deletes the Resource Pool

Parameters:

• pool (str) – Resource pool name

• dc (str) – Datacenter name

• cluster (str) – Cluster name

destroy_vms(vms)

Destroys the VM’s

Parameters:

vms (list) – VM instance list

erase_partition(host, device_path)

Erase the partitions on the disk

Parameters:

• host (vim.HostSystem) – Host instance

• device_path (str) – Device path to erase the partition e.g:”/vmfs/devices/disks/naa.910229801b540c0125ef160f3048faba”

find_datacenter_by_name(datacenter_name)

Fetches the Datacenter

Parameters:

datacenter_name (str) – Name of the Datacenter

Returns:

Datacenter instance

Return type:

vim.Datacenter

find_datastore_by_name(datastore_name, datacenter_name)

Fetches the Datastore

Parameters:

• datastore_name (str) – Name of the Datastore

• datacenter_name (str) – Name of the Datacenter

Returns:

Datastore instance

Return type:

vim.Datastore

find_ip_by_vm(vm, datacenter_name, cluster_name, resource_pool_name)

Fetches the IP for the VM

Parameters:

• vm (str) – Name of VM

• datacenter_name – Name of the Datacenter

• cluster_name – Name of the cluster

• resource_pool_name – Name of the Resource Pool

Returns:

IP of the VM

Return type:

str

find_object_by_name(content, name, obj_type, folder=None, recurse=True)

Finds object by given name

Parameters:

• content (vim.ServiceInstanceContent) – Service Instance Content

• name (str) – Name to search

• obj_type (list) – list of vim.type (e.g: For VM’s, type is vim.VirtualMachine For Hosts, type is vim.HostSystem)

• folder (str) – Folder name

• recurse (bool) – True for recursive search

Returns:

Type of vim instance None: If vim.type doesn’t exists

Return type:

vim.type

find_resource_pool_by_name(resource_pool_name)

Fetches the Resource Pool

Parameters:

resource_pool_name (str) – Name of the Resource Pool

Returns:

Resource Pool instance

Return type:

instance

find_vms_without_ip(name, datacenter_name, cluster_name)

Find all VMs without IP from resource pool

Parameters:

• name (str) – Resource pool name

• datacenter_name (str) – Datacenter name

• cluster_name (str) – vSphere Cluster name

Returns:

VM instances (vim.VirtualMachine)

Return type:

list

get_active_partition(host)

Fetches the active partition disk on Host

Parameters:

host (vim.HostSystem) – Host instance

Returns:

Active partition disk

Return type:

str

get_active_partition_from_mount_info(host)

Gets the active partition from mount info

Parameters:

host (vim.HostSystem) – Host instance

Returns:

Active partition disk

Return type:

str

get_all_objs(content, vimtype, folder=None, recurse=True)

Generate objects of type vimtype

Parameters:

• content (vim.ServiceInstanceContent) – Service Instance Content

• vimtype (vim.type) – Type of vim (e.g: For VM’s, type is vim.VirtualMachine For Hosts, type is vim.HostSystem)

• folder (str) – Folder name

• recurse (bool) – True for recursive search

Returns:

Dictionary of objects and corresponding name

e.g:{

‘vim.Datastore:datastore-12158’: ‘datastore1 (1)’, ‘vim.Datastore:datastore-12157’: ‘datastore1 (2)’ }

Return type:

dict

get_all_vms_in_dc(dc)

Fetches all VMs in Datacenter

Parameters:

dc (str) – Datacenter name

Returns:

List of VMs instance in a Datacenter

Return type:

list

get_all_vms_in_pool(name, dc, cluster)

Gets all VM’s in Resource pool

Parameters:

• name (str) – Resource pool name

• dc (str) – Datacenter name

• cluster (str) – Cluster name

Returns:

VM instances (vim.VirtualMachine)

Return type:

list

Raises:

ResourcePoolNotFound – when Resource pool doesn’t exist

get_cluster(name, dc)

Gets the cluster

Parameters:

• name (str) – Cluster name

• dc (str) – Datacenter name

Returns:

Cluster instance

Return type:

vim.ClusterComputeResource

get_compute_vms_in_pool(name, dc, cluster)

Gets all compute VM’s in Resource pool

Parameters:

• name (str) – Resource pool name

• dc (str) – Datacenter name

• cluster (str) – Cluster name

Returns:

VM instances (vim.VirtualMachine)

Return type:

list

property get_content

Retrieves the content

Returns:

Service Instance Content for Host

Return type:

vim.ServiceInstanceContent

get_controller_for_adding_disk(vm)

Gets the controller for adding disk

Parameters:

vm (vim.VirtualMachine) – VM instance

Returns:

controller instance

get_controllers(vm)

Get the controllers for VM

Parameters:

vm (vim.VirtualMachine) – VM instance

Returns:

list of controllers

Return type:

list

get_datastore_free_capacity(datastore_name, datacenter_name)

Gets the Datastore capacity

Parameters:

• datastore_name (str) – Name of the Datastore

• datacenter_name (str) – Name of the Datacenter

Returns:

Datastore capacity in bytes

Return type:

int

get_datastore_type(datastore)

Gets the Datastore Type

Parameters:

datastore (vim.Datastore) – Datastore instance

Returns:

Datastore type. Either VMFS or vsan

Return type:

str

get_datastore_type_by_name(datastore_name, datacenter_name)

Gets the Datastore Type

Parameters:

• datastore_name (str) – Name of the Datastore

• datacenter_name (str) – Name of the Datacenter

Returns:

Datastore type. Either VMFS or vsan

Return type:

str

get_dc(name)

Gets the Datacenter

Parameters:

name (str) – Datacenter name

Returns:

Datacenter instance

Return type:

vim.Datacenter

get_device_by_key(vm, identifier, key='unit_number')

Get the device by key, and a specific identifier

Parameters:

• vm (vim.VirtualMachine) – VM instance

• identifier (str) – The value of either ‘unit_number’ (Disk unit number to remove), ‘volume_path’ (The volume path in the datastore (i.e, ‘[vsanDatastore] d4210a5e-40ce-efb8-c87e-040973d176e1/control-plane-1.vmdk’), or ‘disk_name’(The disk name (i.e, ‘scsi-36000c290a2cffeb9fcf4a5f748e21909’)

• key (str) – Either ‘unit_number’, ‘volume_path’, or ‘disk_name’

Returns:

The virtual disk device object that

matches the key and the identifier

Return type:

pyVmomi.VmomiSupport.vim.vm.device.VirtualDisk

get_host(vm)

Fetches the Host for the VM. Host where VM resides

Parameters:

vm (vim.VirtualMachine) – VM instance

Returns:

Host instance

Return type:

vim.HostSystem

get_host_obj(host_name)

Fetches the Host object

Parameters:

host_name (str) – Host name

Returns:

Host instance

Return type:

vim.HostSystem

get_lunids(dc)

Fetches the LUN ids from the Datacenter

Parameters:

dc (str) – Datacenter name

Returns:

Dictionary contains host name as key and

values as list lun ids

e.g:{

‘HostName1’: [‘02000000193035e73d534’], ‘HostName2’: [‘020000000060034d43333’] }

Return type:

dict

get_mounted_devices(host, datastore_type='VMFS')

Fetches the available storage devices on Host.

Parameters:

• host (vim.HostSystem) – Host instance

• datastore_type (str) – Type of datastore. Either VMFS or vsan By default, it will take VMFS as datastore type.

Returns:

List of storage devices available for use

Return type:

list

get_mounted_devices_in_vmfs(host)

Fetches the devices which was mounted in VMFS

Parameters:

host (vim.HostSystem) – Host instance

Returns:

List of storage devices which was mounted

Return type:

list

get_mounted_devices_in_vsan(host)

Fetches the devices which was mounted in VSAN

Parameters:

host (vim.HostSystem) – Host instance

Returns:

List of storage devices which was mounted

Return type:

list

get_network_device(ip, dc, label='Network adapter 1', network='VM Network')

Get the network adapter for a VM.

Parameters:

• ip (str) – The IP address of the VM.

• dc (str) – The datacenter name where the VM is located.

• label (str, optional) – The label of the network adapter. Defaults to “Network adapter 1”.

• network (str, optional) – The name of the network. Defaults to “VM Network”.

Returns:

The network adapter, or None if not found.

Return type:

vim.vm.device.VirtualEthernetCard

get_passthrough_enabled_devices(vm)

Fetches the passthrough enabled devices

Parameters:

vm (vim.VirtualMachine) – VM instance

Returns:

List of PciPassthroughInfo

Return type:

list

get_pool(name, dc, cluster)

Gets the Resource pool

Parameters:

• name (str) – Resource pool name

• dc (str) – Datacenter name

• cluster (str) – Cluster name

Returns:

Resource pool instance

Return type:

vim.ResourcePool

property get_search_index

Get the search index

Returns:

Instance of Search Index

Return type:

vim.SearchIndex

get_storage_devices(host)

Fetches all the storage devices in the Host. It excludes the enclosures.

Parameters:

host (vim.HostSystem) – Host instance

Returns:

List of storage devices in Host

Return type:

list

get_storage_object_manger()

Gets the vStorageObjectManager

Returns:

vStorageObjectManager

Return type:

vim.vslm.vcenter.VStorageObjectManager

get_unit_number(vm)

Gets the available unit number for the disk

Parameters:

vm (vim.VirtualMachine) – VM instance

Returns:

available unit number for disk

Return type:

int

get_used_devices(host)

Fetches the used storage devices in Host.

Note: Storage devices may be used in different Resource Pools of OCS clusters.

Parameters:

host (vim.HostSystem) – Host instance

Returns:

List of storage devices used

Return type:

list

get_used_unit_number(vm)

Gets the used unit numbers for a VM

Parameters:

vm (vim.VirtualMachine) – VM instance

Returns:

list of unit numbers

Return type:

list

get_vm_by_ip(ip, dc, vm_search=True)

Gets the VM using IP address

Parameters:

• ip (str) – IP address

• dc (str) – Datacenter name

• vm_search (bool) – Search for VMs if True, Hosts if False

Returns:

VM instance

Return type:

vim.VirtualMachine

get_vm_in_pool_by_name(name, dc, cluster, pool)

Gets the VM instance in a resource pool

Parameters:

• name (str) – VM name

• dc (str) – Datacenter name

• cluster (str) – Cluster name

• pool (str) – pool name

Returns:

VM instances

Return type:

vim.VirtualMachine

get_vm_power_status(vm)

Get the VM power status

Parameters:

vm (vm) – VM object

Returns:

VM power status

Return type:

str

get_vms_ips(vms)

Get VMs IPs

Parameters:

vms (list) – VM (vm) objects

Returns:

VMs IPs

Return type:

list

get_volume_path(volume_id, datastore_name, datacenter_name)

Gets the Volume path

Parameters:

• volume_id (str) – Volume ID

• datastore_name (str) – Name of the Datastore

• datacenter_name (str) – Name of the Datacenter

Returns:

Path to the volume

Return type:

str

get_vslm_id(volume_id, datastore, storage)

Gets the VSLM ID

Parameters:

• volume_id (str) – Volume ID

• datastore (vim.Datastore) – Datastore instance

• storage (vim.vslm.vcenter.VStorageObjectManager) – vStorageObjectManager

Returns:

vim.vslm.ID

Raises:

VSLMNotFoundException – In case VSLM not found

is_resource_pool_exist(pool, dc, cluster)

Check whether resource pool exists in cluster or not

Parameters:

• pool (str) – Resource pool name

• dc (str) – Datacenter name

• cluster (str) – Cluster name

Returns:

True if resource pool exists, otherwise False

Return type:

bool

is_resource_pool_prefix_exist(pool_prefix, dc, cluster)

Check whether or not resource pool with the provided prefix exist

Parameters:

• pool_prefix (str) – The prefix to look for

• dc (str) – Datacenter name

• cluster (str) – Cluster name

Returns:

True if a resource pool with the same name prefix exists, False otherwise

Return type:

bool

is_template_exist(template_name, dc)

Checks whether template exists in Datacenter

Parameters:

• template_name (str) – template name

• dc (str) – Datacenter name

Returns:

True if template exists, otherwise False

Return type:

bool

is_vm_obj_exist(vm)

Check if the vm object exists.

Parameters:

vm (vim.VirtualMachine) – VM instance

Returns:

True if the VM object exists, False otherwise.

Return type:

bool

map_lunids_to_devices(**kwargs)

Maps the LUN ids to storage devices

Parameters:

**kwargs (dict) – Host to LUN mapping e.g: data = get_lunids(dc) map_lunids_to_devices(**data)

Returns:

Dictionary contains host instance as key and

value as list of device path

Return type:

dict

poweroff_vms(vms)

Powers off the VM and wait for operation to complete

Parameters:

vms (list) – VM instance list

poweron_vms(vms)

Powers on the VM and wait for operation to complete

Parameters:

vms (list) – VM instance list

remove_disk(vm, identifier, key='unit_number', datastore=True)

Removes the Disk from VM and datastore. By default, it will delete the disk ( vmdk ) from VM and backend datastore. If datastore parameter is set to False, then it will ONLY removes the disk from VM

Parameters:

• vm (vim.VirtualMachine) – VM instance

• identifier (str) – The value of either ‘unit_number’ (Disk unit number to remove), ‘volume_path’ (The volume path in the datastore (i.e, ‘[vsanDatastore] d4210a5e-40ce-efb8-c87e-040973d176e1/control-plane-1.vmdk’), or ‘disk_name’(The disk name (i.e, ‘scsi-36000c290a2cffeb9fcf4a5f748e21909’)

• key (str) – Either ‘unit_number’, ‘volume_path’, or ‘disk_name’

• datastore (bool) – Delete the disk (vmdk) from backend datastore if True

remove_disks(vm)

Removes all the extra disks for a VM

Parameters:

vm (vim.VirtualMachine) – VM instance

remove_vms_from_inventory(vms)

Remove the VM’s from inventory

Parameters:

vms (list) – VM instance list

restart_vms(vms, force=False)

Restart VMs by VM Reset or Guest reboot

Parameters:

• vms (list) – VM (vm) objects

• force (bool) – True for Hard reboot(VM Reset), False for Soft reboot(Guest Reboot)

restart_vms_by_stop_and_start(vms, force=True)

Stop and Start VMs

Parameters:

• vms (list) – VM (vm) objects

• force (bool) – True for VM ungraceful power off, False for graceful VM shutdown

start_vms(vms, wait=True)

Start VMs

Parameters:

• vms (list) – VM (vm) objects

• wait (bool) – Wait for VMs to start

stop_vms(vms, force=True, wait=True)

Stop VMs

Parameters:

• vms (list) – VM (vm) objects

• force (bool) – True for VM ungraceful power off, False for graceful VM shutdown

• wait (bool) – Wait for the VMs to stop

wait_for_task(task)

Wait for a task to finish

Parameters:

task (instance) – Instance for the task

Returns:

VM instance

Return type:

instance

wait_for_vm_delete(vm, timeout=60)

Wait for the vm object to delete.

Parameters:

• vm (vim.VirtualMachine) – VM instance

• timeout (int) – Time to wait for the VM object to delete.

Returns:

True if the VM object is deleted in the given timeout, False otherwise.

Return type:

bool

ocs_ci.utility.vsphere_nodes module

Module that contains operations related to vSphere nodes in a cluster This module directly interacts with VM nodes

class ocs_ci.utility.vsphere_nodes.VSPHERENode(host, user=None, private_key=None)

Bases: object

A class that handles operations related to VM node

reboot()

Reboots the node

Returns:

tuple which contains command return code, output and error

Return type:

tuple

replace_ntp_server_in_chrony(server=None)

Replace default NTP server to given server

Parameters:

server (str) – NTP server

restart_chrony()

Restarts chrony service

Returns:

True if successful restarts of chrony, False otherwise

Return type:

bool

restart_service(service_name)

Restarts the given service

set_host_name(host_name)

Sets the host name

Parameters:

host_name (str) – Name to set as host name

Returns:

tuple which contains command return code, output and error

Return type:

tuple

ocs_ci.utility.vsphere_nodes.get_node_ips_from_module(module)

Fetches the node IP’s in cluster from terraform state file

Parameters:

module (str) – Module name in terraform.tfstate file e.g: constants.COMPUTE_MODULE

Returns:

List of module node IP’s

Return type:

list

ocs_ci.utility.vsphere_nodes.update_ntp_and_restart_chrony(node, server=None)

Update the NTP in the node and restarts chronyd service

Parameters:

• node (str) – Hostname/IP of node

• server (str) – NTP server to update in chrony config

ocs_ci.utility.vsphere_nodes.update_ntp_compute_nodes()

Updates NTP server on all compute nodes

ocs_ci.utility.workloadfixture module

This module contains functions implementing functionality of workload fixtures in ocs-ci.

ocs_ci.utility.workloadfixture.ignore_next_measurement_file()

Sets config.RUN[“ignore_next_measurement_file”] to True. This is supposed to be used in monitoring tests that are marked as @flaky to make sure that the measurement is done again.

ocs_ci.utility.workloadfixture.is_measurement_done(result_file)

Has the measurement been already performed and stored in a result file?

Returns:

True if the measurement has been already performed.

Return type:

bool

ocs_ci.utility.workloadfixture.measure_operation(operation, result_file, minimal_time=None, metadata=None, measure_after=False, pagerduty_service_ids=None, threading_lock=None)

Get dictionary with keys ‘start’, ‘stop’, ‘metadata’ and ‘result’ that contain information about start and stop time of given function and its result.

Parameters:

• operation (function) – Function to be performed

• result_file (str) – File name that should contain measurement results including logs in json format. If this file exists then it is used for test. Existing file is ignored when config.RUN[“ignore_next_measurement_file”] is set.

• minimal_time (int) – Minimal number of seconds to monitor a system. If provided then monitoring of system continues even when operation is finshed. If not specified then measurement is finished when operation is complete

• metadata (dict) – This can contain dictionary object with information relevant to test (e.g. volume name, operating host, …)

• measure_after (bool) – Determine if time measurement is done before or after the operation returns its state. This can be useful e.g. for capacity utilization testing where operation fills capacity and utilized data are measured after the utilization is completed

• pagerduty_service_ids (list) – Service IDs from PagerDuty system used incidents query

• threading_lock (threading.RLock) – Lock used for synchronization of the threads in Prometheus calls

Returns:

contains information about start and stop time of given

function and its result and provided metadata Example:

{
    'start': 1569827653.1903834,
    'stop': 1569828313.6469617,
    'result': 'rook-ceph-osd-2',
    'metadata': {'status': 'success'},
    'prometheus_alerts': [{'labels': ...}, {...}, ...]
}

Return type:

dict

Module contents