Automatic test system

Description

We need to make scripting to support automated testing from scratch. We have decide to make a BASH framework with Ansible.

We would like to automate:

  • initial board setup (automated 1st login)

  • search for boards from config file, subnet or combination,

  • check for services we define (port 22 for example) and mark those that are not responding but are on the list

  • simple tasks such as upgrade, change to beta repository, downgrade to specific kernel, etc.

  • run various of tasks which can be added without limits

  • run various of benchmarks which can be added without limits

  • run tasks in parallel

  • make reporting in HTML, XML (common and per board)

Most of those ideas are covered in some basic form in our first try: https://github.com/armbian/autotests

More ideas:
https://github.com/SoInteractive/ansible-benchmark

80% Done
Loading...

Checklist

hide

Activity

Show:

Igor Pecovnik November 15, 2021 at 6:40 AM

I hope we will get more help on crushing down those tasks.

pavelmc November 4, 2021 at 8:08 PM

For the Ansible part I’m advanced on the dynamic host detection based on some user input.

Like a whole subnet or csv hosts, etc. It has 3 modes, dynamic (--list), static (--static) and debug

  • dynamic is the way ansible likes, ouptuts json but can be slow if a few hosts on a broad network (~25 sec on a wifi lan /24, on EACH run)

  • static is the fastest, it builds a static host.ini file for ansible

  • debug is verbose and it’s output is for human supervision of the results

Work is done on (push soon) to merge back to armbian/sa

Once it’s pushed I will move to #AR-949 and so on…

 

Any advice is appreciated

Details

Assignee

Reporter

Components

Priority

Epic Name

Created October 27, 2021 at 4:05 PM
Updated November 12, 2023 at 9:41 PM