VOLTS
Voip Open Linear Tester Suite
Functional tests for VoIP systems based on
voip_patrol
and docker.
Some alternative introduction to the system can be found on
DOU(ukrainian)
10'000 ft. view
System is designed to run simple call scenarios, that you usually do
with your desk phones.
Scenarios are run one by one from scenarios folder in
alphabetical order, which could be considered as a limitation, but also
allows you to reuse same accounts in a different set of tests. This
stands for Linear in the name ;) So, call some
destination(s) with one(or more) device(s) and control call arrival on
another phone(s).
But wait, there is more. VOLTS also can integrate with your MySQL and/or
PostgreSQL databases to write some data there before test and remove it
after.
It will make, receive calls and configure database. It's not do
transfers at the moment. Sorry. I don't need it
Suite consists of 4 parts, that are running sequentially
- Preparation - at this part we're transforming templates to real scenarios of voip_patrol using Jinja2 template engine with jinja2_time extension to put some dynamic data based on time.
- Running database scripts. Usually - put some data inside some routing or subscriber data.
- Running voip_patrol against scenario.
- Again running database scripts. Usually - remove data that had been put at the stage 2.
- Report - at this part we're analyzing results of previous step reading and interpreting file obtained running step 3. Printing results in a desired way. Table by default. Steps 2-4 are running sequentially against scenarios files prepared at step 1. One at a time.
Getting
To get it you'll need git installed. But who don't have it nowdays?
Than just run
# git clone https://github.com/igorolhovskiy/volts.git# cd volts
Building
Suite is designed to run locally from your Linux PC or Mac (maybe). And
of course, docker should be installed. It's up to you.
To build, just run
# ./build.sh
inside volts directory. It would build 4 docker images and tag em accordingly.
In a case if voip_patrol is updated, you need to rebuild
it's container again, you can do it with
# ./build.sh -r
Running
After building, just run
# ./run.sh
Simple, isn't it? This will run all scenarios found in scenarios folder one by one. To run single scenario, run
# ./run.sh <scenario_name>
or
# ./run.sh scenarios/<scenario_name>
After running of the suite you can always find a
voip_patrol presented results in
tmp/output folder.
But simply run something blindly is boring, so before this, best to do
some
Configuration
We suppose to configure 2 parts here. First, and most complex are
Scenarios
VOLTS scenarios are combined voip_patrol and
database-controlled scenarios, that are just being templatized with
Jinja2 style. Mostly done not to repeat some passwords,
usernames, domains, etc.
Also, due to using jinja2-time extension, it's possible to
use dynamic time/date values in your scenarios, for example testing some
time-based rules on your PBX. For full documentation on how to use this
type of data, please refer to
'jinja2-time'
documentation. Values for templates are taken from
scenarios/config.yaml
One thing to mention here, that vars from global section
transforms to c. and from accounts to
a. in templates for shorter notation. Also all settings
from global section are inherited to
accounts section automatically, unless they are defined
there explicitly. To get most of it, please refer to
voip_patrol
config, but here just some more basic examples.
config.yaml
global:
domain: '<SOME_DOMAIN>'
transport: 'tls'
srtp: 'dtls,sdes,force'
play_file: '/voice_ref_files/8000_12s.wav'
databases:
'sippproxydb':
type: 'mysql'
user: 'SIP Proxyrw'
password: 'SIP Proxyrwpass'
base: 'SIP Proxy'
host: 'mySIP Proxydb.local'
accounts:
'88881':
username: '88881'
auth_username: '88881-1'
password: 'SuperSecretPass1'
'88882':
username: '88882'
auth_username: '88882-67345'
password: 'SuperSecretPass2'
'90001':
username: '90001'
auth_username: '90001'
password: 'SuperSecretPass3'
Make a successful register
Here we assume that account data is known to our PBX.
<config>
<actions>
<action type="register" label="Register {{ a.88881.label }}"
transport="{{ a.88881.transport }}"
<!-- "account" parameter is more used in receive call on this account later -->
account="{{ a.88881.label }}"
<!-- username would be a part of AOR - <sip:username@realm> -->
username="{{ a.88881.username }}"
<!-- auth_username would be used in WWW-Authorize procedure -->
auth_username="{{ a.88881.auth_username }}"
password="{{ a.88881.password }}"
registrar="{{ c.domain }}"
realm="{{ a.88881.domain }}"
<!-- We're expecting get 200 code here, so REGISTER is successfull -->
expected_cause_code="200"
/>
<!-- Just wait 2 sec for all timeouts -->
<action type="wait" complete="true" ms="2000"/>
</actions>
</config>
.. but what if we need to add account data to PBX dynamically? Assume, that we have SIP Proxy as a PBX here.
<!-- Test simple register -->
<config>
<section type="database">
<actions>
<!-- "sippproxydb" here is referring to entity in "databases" from config.yaml. "stage" is explained a bit below -->
<action database="sippproxydb" stage="pre">
<!-- what data are we gonna insert into "subscriber" table? -->
<table name="subscriber" type="insert" cleanup_after_test="true">
<field name="username" value="{{ a.88881.username }}"/>
<field name="domain" value="{{ c.domain }}"/>
<field name="password" value="{{ a.88881.password }}"/>
</table>
</action>
</actions>
</section>
<section type="voip_patrol">
<actions>
<action type="register" label="Register {{ a.88881.label }}"
transport="{{ a.88881.transport }}"
<!-- "account" parameter is more used in receive call on this account later -->
account="{{ a.88881.label }}"
<!-- username would be a part of AOR - <sip:username@realm> -->
username="{{ a.88881.username }}"
<!-- auth_username would be used in WWW-Authorize procedure -->
auth_username="{{ a.88881.auth_username }}"
password="{{ a.88881.password }}"
registrar="{{ c.domain }}"
realm="{{ a.88881.domain }}"
<!-- We're expecting get 200 code here, so REGISTER is successfull -->
expected_cause_code="200"
/>
<!-- Just wait 2 sec for all timeouts -->
<action type="wait" complete="true" ms="2000"/>
</actions>
</section>
</config>
Database config is also done in XML. We have 2 stages of database scripts.
- pre - Lauched before running voip_patrol. Usually stage to put some accounts data, routing, etc
- post - Obviously, running after voip_patrol. For cleanup data inserted in pre stage.
So, inside database action you specify tables you're working with. Each table secton have 4 attributes.
- name - actually name of table we're working with
- type - could be insert, replace and delete. Forming actually INSERT, REPLACE and DELETE SQL statements for database.
- continue_on_error - optional. Em.. ignore errors on preformed actions and continue no matter what. By default database actions will be stopped after encountering first error.
- cleanup_after_test - optional. Allows you not to write explicit post stage for your insert types. Will automatically form delete type on post stage for all insert (not replace ) that were declared on pre stage.
Expect fail on register
We're deleting data from database and restoring it afterwards.
<!-- Test simple register -->
<config>
<section type="database">
<actions>
<!-- "sippproxydb" here is referring to entity in "databases" from config.yaml. "stage" is explained a bit below -->
<action database="sippproxydb" stage="pre">
<!-- what data are we gonna insert into "subscriber" table? -->
<table name="subscriber" type="insert" cleanup_after_test="true">
<field name="username" value="{{ a.88881.username }}"/>
<field name="domain" value="{{ c.domain }}"/>
<field name="password" value="{{ a.88881.password }}"/>
</table>
</action>
</actions>
</section>
<section type="voip_patrol">
<actions>
<action type="register" label="Register {{ a.88881.label }}"
transport="{{ a.88881.transport }}"
<!-- "account" parameter is more used in receive call on this account later -->
account="{{ a.88881.label }}"
<!-- username would be a part of AOR - <sip:username@realm> -->
username="{{ a.88881.username }}"
<!-- auth_username would be used in WWW-Authorize procedure -->
auth_username="{{ a.88881.auth_username }}"
password="{{ a.88881.password }}"
registrar="{{ c.domain }}"
realm="{{ a.88881.domain }}"
<!-- We're expecting get 200 code here, so REGISTER is successfull -->
expected_cause_code="200"
/>
<!-- Just wait 2 sec for all timeouts -->
<action type="wait" complete="true" ms="2000"/>
</actions>
</section>
</config>
Simple call scenario
Register with 1 account and make a call from 90001 to
88881. Max wait time to answer - 15 sec, duration of
connected call - 10 sec. Point, we don't register account
90001 here, as we're not receiving a calls on it, just need
to provide credentials on INVITE.
Also trick, match_account in accept perfectly
links with account in register.
<config>
<actions>
<!-- As we're using call functionality here - define list of codecs -->
<action type="codec" disable="all"/>
<action type="codec" enable="pcma" priority="250"/>
<action type="codec" enable="pcmu" priority="249"/>
<action type="codec" enable="opus" priority="248"/>
<action type="register" label="Register {{ a.88881.label }}"
transport="{{ a.88881.transport }}"
account="{{ a.88881.label }}"
username="{{ a.88881.username }}"
auth_username="{{ a.88881.auth_username }}"
password="{{ a.88881.password }}"
registrar="{{ c.domain }}"
realm="{{ c.domain }}"
expected_cause_code="200"
<!-- Make sure we're using SRTP on a call receive. This is done here as accounts are created before accept(answer) action -->
srtp="{{ a.88881.srtp }}"
/>
<action type="wait" complete="true" ms="2000"/>
<action type="accept" label="Receive call on {{ a.88881.label }}"
<!-- This is not a load test - so only 1 call is expected -->
call_count="1"
<!-- Make sure we're received a call on a previously registered account -->
match_account="{{ a.88881.label }}"
<!-- Hangup in 10 seconds after answer -->
hangup="10"
<!-- Send back "200 OK" -->
code="200" reason="OK"
transport="{{ a.88881.transport }}"
<!-- Make sure we're using SRTP -->
srtp="{{ a.88881.srtp }}"
<!-- Play some file back to gather RTCP stats in report -->
play="{{ c.play_file }}"
/>
<action type="call" label="Call {{ a.90001.label }} -> {{ a.88881.label }}"
transport="tls"
<!-- We're waiting for an answer -->
expected_cause_code="200"
caller="{{ a.90001.label }}@{{ c.domain }}"
callee="{{ a.88881.label }}@{{ c.domain }}"
from="sip:{{ a.90001.label }}@{{ c.domain }}"
to_uri="{{ a.88881.label }}@{{ c.domain }}"
max_duration="20" hangup="10"
<!-- We're specifying all auth data here for INVITE -->
auth_username="{{ a.90001.username }}"
password="{{ a.90001.password }}"
realm="{{ c.domain }}"
rtp_stats="true"
max_ring_duration="15"
srtp="{{ a.90001.srtp }}"
play="{{ c.play_file }}"
/>
<action type="wait" complete="true" ms="30000"/>
</actions>
</config>
Advanced call scenario
Register with 2 accounts and call from third one, not answer on 1st and
make sure we receive call on second. So, your PBX should be configured
to make a Forward-No-Answer from 88881 to
88882.
Also make sure, that on 88882 we got the call from
90001 (based on CallerID).
<config>
<actions>
<action type="codec" disable="all"/>
<action type="codec" enable="pcma" priority="250"/>
<action type="codec" enable="pcmu" priority="249"/>
<action type="codec" enable="opus" priority="248"/>
<action type="register" label="Register {{ a.88881.label }}"
transport="{{ a.88881.transport }}"
account="{{ a.88881.label }}"
username="{{ a.88881.username }}"
auth_username="{{ a.88881.auth_username }}"
password="{{ a.88881.password }}"
registrar="{{ c.domain }}"
realm="{{ c.domain }}"
expected_cause_code="200"
srtp="{{ a.88881.srtp }}"
/>
<action type="register" label="Register {{ a.88882.label }}"
transport="{{ a.88882.transport }}"
account="{{ a.88882.label }}"
username="{{ a.88882.username }}"
auth_username="{{ a.88882.auth_username }}"
password="{{ a.88882.password }}"
registrar="{{ c.domain }}"
realm="{{ c.domain }}"
expected_cause_code="200"
srtp="{{ a.88882.srtp }}"
/>
<action type="wait" complete="true" ms="2000"/>
<action type="call" label="Call from 90001 to 88881->88882"
transport="{{ a.90001.transport }}"
expected_cause_code="200"
caller="{{ a.90001.label }}@{{ c.domain }}"
callee="88881@{{ c.domain }}"
from="sip:{{ a.90001.label }}@{{ c.domain }}"
to_uri="88881@{{ c.domain }}"
max_duration="20" hangup="10"
auth_username="{{ a.90001.username }}"
password="{{ a.90001.password }}"
realm="{{ c.domain }}"
rtp_stats="true"
<!-- Set some high ring timeout, so delayed forward will happen -->
max_ring_duration="60"
srtp="{{ a.90001.srtp }}"
play="{{ c.play_file }}"
/>
<action type="accept" label="Receive call on {{ a.88881.label }}"
match_account="{{ a.88881.label }}"
call_count="1"
hangup="10"
ring_duration="30"
<!-- We're expecting a CANCEL here. And it's not optional -->
cancel="force"
transport="{{ a.88881.transport }}"
srtp="{{ a.88881.srtp }}"
/>
<action type="accept" label="Receive call on {{ a.88882.label }}"
match_account="{{ a.88882.label }}"
call_count="1"
hangup="10"
code="200" reason="OK"
transport="{{ a.88882.transport }}"
srtp="{{ a.88882.srtp }}"
play="{{ c.play_file }}">
<!-- Check that From header matching what we need. This way we can control CallerID. Adjust domain (and whole regex) accordingly -->
<check-header name="From" regex="^.*sip:{{ a.90001.label }}@example\.com>.*$"/>
</action>
<action type="wait" complete="true" ms="20000"/>
</actions>
</config>
Advanced call scenario - 2. Now with databases.
Schema - SIP Proxy subscriber and than we have Asterisk behind as PBX.
config.yaml
global:
domain: '<SOME_DOMAIN>'
transport: 'tls'
srtp: 'dtls,sdes,force'
play_file: '/voice_ref_files/8000_2m30.wav'
asterisk_context: 'default'
databases:
'sippproxydb':
type: 'mysql'
user: 'SIP Proxyrw'
password: 'SIP Proxyrwpass'
base: 'SIP Proxy'
host: 'mySIP Proxydb.local'
'astdb':
type: 'pgsql'
user: 'asteriskrw'
password: 'asteriskrwpass'
base: 'asterisk'
host: 'myasteriskdb.local'
accounts:
'90011':
username: '90011'
password: 'SuperSecretPass1'
ha1: 'SuperSecretHA1'
'90012':
username: '90012'
password: 'SuperSecretPass2'
ha1: 'SuperSecretHA2'
And now we need to populate all databases and make a call!
<!-- Register with 90012 and receive a call from 90011 -->
<config>
<section type="database">
<actions>
<!-- add subscribers to SIP Proxy -->
<action database="sippproxydb" stage="pre">
<table name="subscriber" type="insert" cleanup_after_test="true">
<field name="username" value="{{ a.90011.username }}"/>
<field name="domain" value="{{ c.domain }}"/>
<field name="ha1" value="{{ a.90011.ha1 }}"/>
<!-- here password due to ha1 is useless, so we can put some data based on jinja2_time.TimeExtension (https://github.com/hackebrot/jinja2-time) -->
<field name="password" value="{% now 'local' %}"/>
</table>
<table name="subscriber" type="insert" cleanup_after_test="true">
<field name="username" value="{{ a.90012.username }}"/>
<field name="domain" value="{{ c.domain }}"/>
<field name="ha1" value="{{ a.90012.ha1 }}"/>
<field name="password" value="{% now 'local' + 'days=1', '%D' %}"/>
</table>
</action>
<!-- add endpoints and aors to Asterisk -->
<action database="astdb" stage="pre">
<table name="ps_endpoints" type="insert" cleanup_after_test="true">
<field name="id" value="{{ a.90011.label }}"/>
<field name="transport" value="transport-udp"/>
<field name="aors" value="{{ a.90011.label }}"/>
<field name="context" value="{{ c.asterisk_context }}"/>
<field name="disallow" value="all"/>
<field name="allow" value="!all,opus,alaw"/>
<field name="direct_media" value="no"/>
<field name="ice_support" value="no"/>
<field name="rtp_timeout" value="3600"/>
</table>
<table name="ps_aors" type="insert" cleanup_after_test="true">
<field name="id" value="{{ a.90011.label }}"/>
<field name="contact" value="sip:{{ a.90011.label }}@{{ c.domain }}:5060"/>
</table>
<table name="ps_endpoints" type="insert" cleanup_after_test="true">
<field name="id" value="{{ a.90012.label }}"/>
<field name="transport" value="transport-udp"/>
<field name="aors" value="{{ a.90012.label }}"/>
<field name="context" value="{{ c.asterisk_context }}"/>
<field name="disallow" value="all"/>
<field name="allow" value="!all,opus,alaw"/>
<field name="direct_media" value="no"/>
<field name="ice_support" value="no"/>
<field name="rtp_timeout" value="3600"/>
</table>
<table name="ps_aors" type="insert" cleanup_after_test="true">
<field name="id" value="{{ a.90012.label }}"/>
<field name="contact" value="sip:{{ a.90012.label }}@{{ c.domain }}:5060"/>
</table>
</action>
</actions>
</section>
<section type="voip_patrol">
<!-- Make a call from one endpoint to other -->
<actions>
<action type="codec" disable="all"/>
<action type="codec" enable="pcma" priority="250"/>
<action type="codec" enable="pcmu" priority="249"/>
<action type="codec" enable="opus" priority="248"/>
<action type="register" label="Register {{ a.90012.label }}"
transport="{{ a.90012.transport }}"
account="{{ a.90012.username }}"
username="{{ a.90012.label }}"
auth_username="{{ a.90012.username }}"
password="{{ a.90012.password }}"
registrar="{{ c.domain }}"
realm="{{ c.domain }}"
expected_cause_code="200"
srtp="{{ a.90012.srtp }}"
/>
<action type="wait" complete="true" ms="2000"/>
<action type="accept" label="Receive call on {{ a.90012.label }} from {{ a.90011.label }}"
call_count="1"
match_account="{{ a.90012.username }}"
hangup="10"
code="200" reason="OK"
transport="{{ a.90012.transport }}"
srtp="{{ a.90012.srtp }}"
play="{{ c.play_file }}">
<check-header name="From" regex="^.*sip:{{ a.90011.label }}@.*$"/>
</action>
<action type="call" label="Call {{ a.90011.label }} -> {{ a.90012.label }}"
transport="tls"
expected_cause_code="200"
caller="{{ a.90011.label }}@{{ c.domain }}"
callee="{{ a.90012.label }}@{{ c.domain }}"
from="sip:{{ a.90011.label }}@{{ c.domain }}"
to_uri="{{ a.90012.label }}@{{ c.domain }}"
max_duration="20" hangup="10"
auth_username="{{ a.90011.username }}"
password="{{ a.90011.password }}"
realm="{{ c.domain }}"
rtp_stats="true"
max_ring_duration="15"
srtp="{{ a.90011.srtp }}"
play="{{ c.play_file }}"
/>
<action type="wait" complete="true" ms="30000"/>
</actions>
</section>
</config>
run.sh script
Not that much to configure here, mostly you'll be interested in setting environement variables at the start of the script
| Variable name | Description |
| REPORT_TYPE | Actually, report type, that would be provided at the end. table - print results in table, only failed tests are pritend. json - print results in JSON format, only failed tests are pritend. table_full, json_full - prints results in table or JSON respectively, but print full info on tests passed |
| VP_LOG_LEVEL | voip_patrol log level on the console |
Results
As a results, you will have table like
+------------------------------------+-----------------------------------------------------------+--------+------------------+
| Scenario | Test | Status | Text |
+------------------------------------+-----------------------------------------------------------+--------+------------------+
| 01-register | | PASS | Scenario passed |
| | Register 88881-00001 | PASS | Main test passed |
| 02-call-echo | | PASS | Scenario passed |
| | Call to 11111 (echo) | PASS | Main test passed |
| 03-register-wait-for-call-1 | | PASS | Scenario passed |
| | Register 88881-00001 | PASS | Main test passed |
| | Call to ##88881 from 88882 | PASS | Main test passed |
| | default | PASS | Main test passed |
| 04-register-wait-for-call-2 | | PASS | Scenario passed |
| | Register 90002-00002 | PASS | Main test passed |
| | Call 90001 -> 90002 | PASS | Main test passed |
| | Receive call on 90002-00002 from 90001 | PASS | Main test passed |
| 05-immediate-call-forward | | PASS | Scenario passed |
| | Register 90002-00002 | PASS | Main test passed |
| | Call from 90001 to 91002->90002 | PASS | Main test passed |
| | Receive call on 90002-00002 | PASS | Main test passed |
....
| 37-team-call-from-paused-member | | PASS | Scenario passed |
| | Register 90003-20614 | PASS | Main test passed |
| | Register 90001-22466 | PASS | Main test passed |
| | Register 90002-00002 | PASS | Main test passed |
| | Receive call on 90003-20614 and CANCEL | PASS | Call canceled |
| | Call from team member 90001 -> 90543 | PASS | Main test passed |
| | Receive call on 90002-00002 and answer | PASS | Main test passed |
+------------------------------------+-----------------------------------------------------------+--------+------------------+
All scenarios are OK!
Not really much to describe here, just read info on the console