Test Failure Classifier Connector
Description
tfacon is a CLI tool for connecting Test Management Platforms and Test Failure Analysis Classifier. Test Failure Analysis Classifier is an AI/ML predictioner developed by Red Hat D&O Data Science Team which can predict the test's catalog. It supports AutomationBug, ProductBug, SystemBug on Report Portal now. tfacon only support report portal at this moment. We will support more platforms in the future.
User Guide
Installation
Via go get
go get -u github.com/RedHatQE/tfacon@master
Via pip
pip install tfacon
Get Started
tfacon.yml
This is where you store all parameters like this You must use auth_token from a superadmin account to run tfacon, otherwise the validation will fail!
launch_id: "your launch_id goes here"
project_name: "your project name goes here"
auth_token: "xxxxx-xxxx-xxxxx-xxxx-xxxxx"
platform_url: "https://reportportal.com"
tfa_url: "your tfa classifier url"
connector_type: "RPCon"
The default name is tfacon.yml, you can change this by edit this environment variable TFACON_YAML_PATH
tfacon.cfg
This is where you put all the config information for tfacon
[config]
concurrency=true
add_attributes=true
The default name for this cfg file is tfacon.cfg, you can change this by edit this environment variable TFACON_CONFIG_PATH
list
❯ tfacon list -h
list all information constructed from tfacon.yml/enviroment variables/cli
Usage:
tfacon list [flags]
Flags:
--auth-token string The AUTH_TOKEN of report portal
--connector-type string The type of connector you want to use(example: RPCon, PolarionCon, JiraCon) (default "RPCon")
-h, --help help for list
--launch-id string The launch id of report portal
--platform-url string The url to the test platform(example: https://reportportal-ccit.apps.ocp4.prod.psi.redhat.com) (default "default val for platform url")
--project-name string The project name of report portal
--tfa-url string The url to the TFA Classifier (default "default val for tfa url")
Output Example:
❯ tfacon list
--------------------------------------------------
tfacon 1.0.0
Copyright (C) 2021, Red Hat, Inc.
-------------------------------------------------
2021/08/06 03:21:33 Printing the constructed information
LaunchID: 968
ProjectName: TFA_RP
AuthToken: xxxx-xxxx-xxxxxxx-xxxxxx-xxxxxxxxx
RPURL: https://reportportal.com
Client: &{<nil> <nil> <nil> 0s}
TFAURL: https://tfa.com
run
❯ tfacon run -h
run the info retrival and get the pridiction from TFA
Usage:
tfacon run [flags]
Flags:
--auth-token string The AUTH_TOKEN of report portal
--connector-type string The type of connector you want to use(example: RPCon, PolarionCon, JiraCon) (default "RPCon")
-h, --help help for run
--launch-id string The launch id of report portal
--platform-url string The url to the test platform(example: https://reportportal-ccit.apps.ocp4.prod.psi.redhat.com) (default "default val for platform url")
--project-name string The project name of report portal
--tfa-url string The url to the TFA Classifier (default "default val for tfa url")
Example Output
❯ tfacon run --project-name "project_name" --launch-id 1006
2021/08/06 03:46:59 Getting prediction of test item(id): 54799
2021/08/06 03:46:59 Getting prediction of test item(id): 54900
2021/08/06 03:46:59 Getting prediction of test item(id): 54106
2021/08/06 03:46:59 Getting prediction of test item(id): 54555
2021/08/06 03:46:59 Getting prediction of test item(id): 54986
2021/08/06 03:46:59 Getting prediction of test item(id): 54411
2021/08/06 03:46:59 Getting prediction of test item(id): 54824
2021/08/06 03:46:59 Getting prediction of test item(id): 54642
2021/08/06 03:46:59 Getting prediction of test item(id): 54841
This is the return info from update: [{"issueType":"ab001","comment":"","autoAnalyzed":false,"ignoreAnalyzer":false,"externalSystemIssues":[]},{"issueType":"ab001","comment":"","autoAnalyzed":false,"ignoreAnalyzer":false,"externalSystemIssues":[]},{"issueType":"ab001","comment":"Should be marked with custom defect type","autoAnalyzed":false,"ignoreAnalyzer":false,"externalSystemIssues":[]},{"issueType":"si001","comment":"","autoAnalyzed":false,"ignoreAnalyzer":false,"externalSystemIssues":[]},{"issueType":"ab001","comment":"Should be marked with custom defect type","autoAnalyzed":false,"ignoreAnalyzer":false,"externalSystemIssues":[]},{"issueType":"ab001","comment":"Should be marked with custom defect type","autoAnalyzed":false,"ignoreAnalyzer":false,"externalSystemIssues":[]},{"issueType":"ab001","comment":"Should be marked with custom defect type","autoAnalyzed":false,"ignoreAnalyzer":false,"externalSystemIssues":[]},{"issueType":"ab001","comment":"Should be marked with custom defect type","autoAnalyzed":false,"ignoreAnalyzer":false,"externalSystemIssues":[]},{"issueType":"pb001","comment":"Should be marked with custom defect type","autoAnalyzed":false,"ignoreAnalyzer":false,"externalSystemIssues":[]}]
validate
You must use auth_token from a superadmin account to run tfacon, otherwise the validation will fail!
❯ tfacon validate -h
validate if the parameter is valid and if the urls are accesible
Usage:
tfacon validate [flags]
Flags:
--auth-token string The AUTH_TOKEN of report portal
--connector-type string The type of connector you want to use(example: RPCon, PolarionCon, JiraCon) (default "RPCon")
-h, --help help for validate
--launch-id string The launch id of report portal
--platform-url string The url to the test platform(example: https://reportportal-ccit.apps.ocp4.prod.psi.redhat.com) (default "default val for platform url")
--project-name string The project name of report portal
--tfa-url string The url to the TFA Classifier (default "default val for tfa url")
Global Flags:
-v, --verbose You can add this tag to print more detailed info
Example Output
❯ tfacon validate --project-name "TFACON" --launch-id 231 LaunchID: 231 ProjectName: TFACON AuthToken: xxxxxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx RPURL: https://reportportal-dev.com Client: &{<nil> <nil> <nil> 0s} TFAURL: https://tfa.com/latest/model Validation Passed!
You can also add -v to have more detailed information for validation error
init
❯ tfacon init -h
init will create a sample workspace for tfacon
Usage:
tfacon init [flags]
Flags:
-h, --help help for init
Global Flags:
-v, --verbose You can add this tag to print more detailed info
Advanced Config
You can set up advanced config in ./tfacon.cfg by default or you can define the location of the config file with TFACON_CONFIG_PATH environment variable
Example
[config]
concurrency=True
add_attributes=true
Set Concurrency
You can set this to True or False, if you set it to True tfacon will deal with the test items in an async non-blocking way(faster), you can disable it to have a more clear view, but you will have a slower run compared to setting it to True
add_attributes
You can enable this to add an extra attribute "AI Prediction" to all the test items, the value of this attribute will be the prediction extracted from TFA Classifier
Developer Guide
Archietrcue
UML graph
Contributor Guide
Branch name
develop is the development branch master is the stable branch
