Home » Eclipse Projects » Eclipse Titan » Porting 3gpp test suites to Titan
| Porting 3gpp test suites to Titan [message #1787059] |
Fri, 18 May 2018 07:55  |
|
Dear all,
we will examine below the portability of 3gpp-published test suites to Titan.
But before getting started, let me reveal some problems one may encounter with certain Titan releases when
compiling test suites containing large ASN.1 files.
Roughly a year ago, post 6.1.0 release, code to support concatenation of templates was added to Titan:
Bug 512878 - Concatenation for templates
https://bugs.eclipse.org/bugs/show_bug.cgi?id=512878
Unfortunately this had the side effect of dramatically slowing down compilation of some ASN.1 files and this was not caught by our CI cycle.
Now 3gpp test suites are building upon such large files, one being especially large, the UTRAN_RRC_ASN1_Definitions.asn , of 1.46 Megabytes in source.
With this file the slowdown is particularly observable, probably grinding compilation to a halt.
Even if the slicing feature of the compiler ( -U 'number) ' is used, this file will cause problems.
The problem persisted throughout 6.2.0 and 6.3.0 but we caught it before releasing 6.4.0 so the latest source code and the to-be-released 6.4.0 already contain the fix for the problem.
For this exercise I have used a Titan built from the latest source, release candidate for 6.4.0.
The referring bug report is here:
Bug 534678 - Performance issues caused by template concatenation feature
https://bugs.eclipse.org/bugs/show_bug.cgi?id=534678
and the github commit , for those interested in details, here:
Changed handling of optional fields in template concatenation to improve performance (bug 534678)
https://github.com/eclipse/titan.core/commit/736c4bdaa0eaa2709e8292746e2456117d08efdf
The takeaway from this is that for 3gpp test suites one should use either 6.1.0 or , if he/she needs the latest features, 6.4.0 ,
and should avoid 6.2.0 and 6.3.0. However, given the presence of large ASN.1 files we keep recommending using slicing the files -as exemplified below-
so one can benefit from compiling in parallel a higher number of smaller files. This will also have a beneficial effect on memory consumption which will prove to be
a scarce resource when compiling these suites.
For other test suites this slowdown may or may not be observable , but will probably stay under the observation threshold.
All right then, now to the point.
3gpp published test suites are available here:
http://www.ttcn-3.org/index.php/downloads/publicts/publicts-3gpp
and today we will look into UTRA test suites:
http://www.ttcn-3.org/index.php/downloads/publicts/publicts-3gpp/78-3gpp-utra-ue-test-suite
The entire series is published here:
ftp://ftp.3gpp.org/Specs/archive/34_series/34.123-3/
and I have downloaded the latest one:
ftp://ftp.3gpp.org/Specs/archive/34_series/34.123-3/34123-3-e20.zip
dated 4/3/18
This package contains two independent suites, 34123-3-e20_TTCN3_UTRAN and 34123-3-e20_TTCN3_SSNITZ.
What can we expect based on our previous experience of porting ETSI test suites ?
(see posts in this forum)
Of course we will encounter the unholy trinity of: 1) test port declarations 2) codecs 3) external functions ,
that is unstandardised code that Titan expects to be supplied in form of external C++ code.
As before the aim is to make the test suite compilable and not executable.
Whoever wants to take this next step as well will have to add the external code mentioned before.
Let's start with 34123-3-e20_TTCN3_SSNITZ first ( see attached archive)
1. First we should generate a Makefile; here I recommend using Runtime 2 , the function test runtime, which , although is somewhat slower when compiling , is richer in features.
Also , code slicing is strongly recommended.
Here you have the significant lines in my Makefile:
# Flags for the C++ preprocessor (and makedepend as well):
CPPFLAGS = -D$(PLATFORM) -I$(TTCN3_DIR)/include -DTITAN_RUNTIME_2
# Flags for the TTCN-3 and ASN.1 compiler:
COMPILER_FLAGS = -L -w -R -U 8 -M -d
# Execution mode: (either ttcn3 or ttcn3-parallel)
TTCN3_LIB = ttcn3-rt2-parallel
with the meaning of switches:
-M: allow 'omit' in template value lists (legacy behavior)
-d: treat default fields as omit
-R: use function test runtime (TITAN_RUNTIME_2)
-L: add source line info for logging
-U none|type|'number': select code splitting mode for the generated C++ code
-w: suppress warnings
Even if slicing is done in 8 parts, compilation can be done with less processors involved: make -j4 or even make -j2 or simply make.
Please bear in mind that if we engage a number of processors in parallel compilation , the amount of required memory will multiply as well , so make sure you have sufficient memory,
else you may end up with a surprise.
2. the next step we need to take is to add a dummy encoding instruction to the ComponentsType.ttcn:
type set of Component Components; /* @status APPROVED (POS, SSNITZ) */
} // end of group SS_Asn_Entry
with {encode "Some_EncRule"}; //FIXME!!!
This dummy declaration and all other such with { encode "somethinghere"} declarations will have to be added substance later
by adding the appropriate external codecs.
3. All ports in all declarations will be turned internal ;
of course , eventually ports that are mapped to SUT will have to be re-qualified external and the pertinent test port code will have to be added.
4. External functions
I have replaced external function declarations with matching internal but dummy declarations:
CommonDefs.ttcn
/* external */ function fx_KeyDerivationFunction(KDF_Type p_KDF,
bitstring p_Key, //@sic R5-155058 sic@
octetstring p_String) return B256_Type {
return '1111111100000000111111110000000011111111000000001111111100000000111111110000000011111111000000001111111100000000111111110000000011111111000000001111111100000000111111110000000011111111000000001111111100000000111111110000000011111111000000001111111100000000'B
};
/* external */ function fx_GetSystemTime(out Struct_tm_Type p_Struct_tm, /* p_Struct_tm returns local system time;
* C implementation:
* time_t v_Now = time(NULL);
* struct tm *v_Tm = localtime(&v_Now);
*/
out integer p_TimezoneInfo)
{};
With these changes the suite will compile.
Let's move to 34123-3-e20_TTCN3_UTRAN then:
1. We can use the same Makefile settings:
# Flags for the C++ preprocessor (and makedepend as well):
CPPFLAGS = -D$(PLATFORM) -I$(TTCN3_DIR)/include -DTITAN_RUNTIME_2
# Flags for the TTCN-3 and ASN.1 compiler:
COMPILER_FLAGS = -L -w -R -U 8 -M -d
# Execution mode: (either ttcn3 or ttcn3-parallel)
TTCN3_LIB = ttcn3-rt2-parallel
2. Codecs will not give us a headache at this stage but of course will have to be added eventually.
3. All ports will have to be declared internal
4. External functions are to be replaced with dummies:
CommonDefs.ttcn
/* external */ function fx_KeyDerivationFunction(KDF_Type p_KDF,
bitstring p_Key, //@sic R5-155058 sic@
octetstring p_String) return B256_Type {
return '1111111100000000111111110000000011111111000000001111111100000000111111110000000011111111000000001111111100000000111111110000000011111111000000001111111100000000111111110000000011111111000000001111111100000000111111110000000011111111000000001111111100000000'B
};
/* external */ function fx_GetSystemTime(out Struct_tm_Type p_Struct_tm, /* p_Struct_tm returns local system time;
* C implementation:
* time_t v_Now = time(NULL);
* struct tm *v_Tm = localtime(&v_Now);
*/
out integer p_TimezoneInfo)
{};
EUTRA_NB_CommonDefs.ttcn
/*external*/ function fx_CalculateFCS32 (bitstring p_TMSI) return B32_Type {
return '11111111000000001111111100000000'B
};
EUTRA_NB_SecurityDefinitionsAndExternalFunctions.ttcn
/* external */ function fx_NasIntegrityAlgorithm(octetstring p_EncodedNasPdu,
B3_Type p_IntegrityAlgorithm,
B128_Key_Type p_KNASint,
NasCount_Type p_NasCount,
B5_Type p_BearerId, /* @sic R5-101050: BearerId changed to 5 bits sic@ */
MAC_Direction_Type p_Direction) return MessageAuthenticationCode
{
return 'FF00FF00'O
}; /* @status APPROVED (LTE) */
/*external*/ function fx_NasCiphering(octetstring p_EncodedNasPdu,
B3_Type p_CipheringAlgorithm,
B128_Key_Type p_KNASenc,
NasCount_Type p_NasCount,
B5_Type p_BearerId) return octetstring { return ''O } ; /* @sic R5-101050: BearerId changed to 5 bits sic@
@status APPROVED (LTE) */
/*external*/ function fx_NasDeciphering(octetstring p_CipheredNasMsg,
B3_Type p_CipheringAlgorithm,
B128_Key_Type p_KNASenc,
NasCount_Type p_NasCount,
B5_Type p_BearerId) return octetstring { return ''O } ; /* @sic R5-101050: BearerId changed to 5 bits sic@
With this , the UTRAN suite will compile.
One may notice that lot less modifications are needed here that were for the ETSI suites. The reason is that 3gpp uses Titan, in parallel with other commercial
tools to validate their test suites during development.
Best regards
Elemer
[Updated on: Thu, 24 May 2018 16:13] Report message to a moderator |
|
|
| Re: Porting 3gpp test suites to Titan [message #1831418 is a reply to message #1787059] |
Wed, 19 August 2020 11:43  |
 Mujeeb Ur Rehman
Messages: 16
Registered: January 2020 |
Junior Member |
|
|
Hi,
I was able to compile the 3GPP NBIOT Test suite and was also able to execute the test case. Now I am stuck and need to work with PORTS. I need some guidance and Ref to understand the working. Here is the PORT code which has been generated after the Compilation using Titan,
// This Test Port skeleton source file was generated by the
// TTCN-3 Compiler of the TTCN-3 Test Executor version CRL 113 200/6 R5A
// for () on Tue Aug 18 17:14:46 2020
// Copyright (c) 2000-2018 Ericsson Telecom AB
// You may modify this file. Complete the body of empty functions and
// add your member functions here.
#include "NBIOT_SRB_PORT.hh"
namespace NBIOT__ASP__SrbDefs {
NBIOT__SRB__PORT::NBIOT__SRB__PORT(const char *par_port_name)
: NBIOT__SRB__PORT_BASE(par_port_name)
{
}
NBIOT__SRB__PORT::~NBIOT__SRB__PORT()
{
}
void NBIOT__SRB__PORT::set_parameter(const char * /*parameter_name*/,
const char * /*parameter_value*/)
{
}
/*void NBIOT__SRB__PORT::Handle_Fd_Event(int fd, boolean is_readable,
boolean is_writable, boolean is_error) {}*/
void NBIOT__SRB__PORT::Handle_Fd_Event_Error(int /*fd*/)
{
}
void NBIOT__SRB__PORT::Handle_Fd_Event_Writable(int /*fd*/)
{
}
void NBIOT__SRB__PORT::Handle_Fd_Event_Readable(int /*fd*/)
{
}
/*void NBIOT__SRB__PORT::Handle_Timeout(double time_since_last_call) {}*/
void NBIOT__SRB__PORT::user_map(const char * /*system_port*/)
{
}
void NBIOT__SRB__PORT::user_unmap(const char * /*system_port*/)
{
}
void NBIOT__SRB__PORT::user_start()
{
}
void NBIOT__SRB__PORT::user_stop()
{
}
void NBIOT__SRB__PORT::outgoing_send(const NB__SRB__COMMON__REQ& /*send_par*/)
{
}
} /* end of namespace */
////////////////////// .hh File
// This Test Port skeleton header file was generated by the
// TTCN-3 Compiler of the TTCN-3 Test Executor version CRL 113 200/6 R5A
// for () on Tue Aug 18 17:14:46 2020
// Copyright (c) 2000-2018 Ericsson Telecom AB
// You may modify this file. Add your attributes and prototypes of your
// member functions here.
#ifndef NBIOT__SYSTEM__PORT_HH
#define NBIOT__SYSTEM__PORT_HH
#include "NBIOT_ASP_TypeDefs.hh"
namespace NBIOT__ASP__TypeDefs {
class NBIOT__SYSTEM__PORT : public NBIOT__SYSTEM__PORT_BASE {
public:
NBIOT__SYSTEM__PORT(const char *par_port_name = NULL);
~NBIOT__SYSTEM__PORT();
void set_parameter(const char *parameter_name,
const char *parameter_value);
private:
/* void Handle_Fd_Event(int fd, boolean is_readable,
boolean is_writable, boolean is_error); */
void Handle_Fd_Event_Error(int fd);
void Handle_Fd_Event_Writable(int fd);
void Handle_Fd_Event_Readable(int fd);
/* void Handle_Timeout(double time_since_last_call); */
protected:
void user_map(const char *system_port);
void user_unmap(const char *system_port);
void user_start();
void user_stop();
void outgoing_send(const NB__SYSTEM__CTRL__REQ& send_par);
};
} /* end of namespace */
#endif
Can you guide me how i can use outgoing_send and incoming_message to communicate in a loopback form and to transfer Message to UE?
Thanks & Regards,
Mujeeb
|
|
|
| Re: Porting 3gpp test suites to Titan [message #1831431 is a reply to message #1831418] |
Wed, 19 August 2020 16:35 |
|
Hi Mujeeb,
there is no simple and unequivocal answer to your question. Here's why:
The architecture of a TTCN-3 test environment looks like below:
+---------------------+ +--------------------+
| | | |
| TE | | |
| | | |
+---------------------+ | |
| | | |
| | | SUT |
| SA | | |
| | | |
| | | |
| | | |
+---------------------+ +--------------------+
| ^
| |
+---------------------------------+
where TE is the TTCN-3 executable, SUT is the system under test and SA is the system adaptor. The SA contains the protocol layers that are not covered by the TE and are needed to connect the TE to SUT.
A Titan test port is just glue code which connects the TE to the protocol layers in SA. For instance , the Titan TCP test port connect to the IP stack in the OS; the test port plus the IP stack covers the concept of SA.
How should the SA look like in your specific case of NB-IoT Ue conformance testing ? To answer this I will refer to the technical specification
LTE;
Evolved Universal Terrestrial Radio Access (E-UTRA) and
Evolved Packet Core (EPC);
User Equipment (UE) conformance specification;
Part 3: Test suites
(3GPP TS 36.523-3 version 15.3.0 Release 15)
available at: https://www.etsi.org/deliver/etsi_ts/136500_136599/13652303/15.03.00_60/ts_13652303v150300p.pdf
with emphasis on ch. 4A NB-IoT system architecture and test models ,
especially Figure 4A.2.2-1: Test model for NB-IoT RRC/NAS testing ,
and ch. 6.5 NB-IoT ASP Definitions
Figure 6.5-1: NB-IoT ASP Test Model
and also the TTCN-3 code available at
ftp://ftp.3gpp.org/Specs/archive/36_series/36.523-3/
(different versions, I have chosen a newer one)
From these is apparent that your SA (assuming that the SUT is Ue and the TTCN-3 code simulates the behavior of the RAN) should cover the following protocol layers:
+---------------------+ +--------------------+
| | | |
| TE | | |
| | | |
+---------------------+ | |
| RRC | | |
| PDCP | | SUT |
| RLC | | |
| MAC | | |
| PHY | | |
| | | |
+----------+----------+ +-------+------------+
| ^
| |
+---------------------------------+
in which case we are probably talking about some HW with the appropriate radio capabilities, running some SW that can process messages from the TE.
The Titan test port, again, is just some glue code that connects Titan to this SA. In this case the TE+SA connects to Ue over the radio interface.
A simplified setup is usually possible, which in our case might mean bypassing some protocol layers and connect e.g. the RRC layer in SA directly to the RRC layer of the Ue:
+---------------------+ +--------------------+
| | | |
| TE | | |
| | | |
+---------------------+ | |
| | | |
| | | SUT |
| RRC | | |
| | | |
| | | |
| | | RRC |
+----------+----------+ +-------+------------+
| ^
| |
+---------------------------------+
but anything between the two is possible.
Now, if we look closely on Figure 6.5-1: NB-IoT ASP Test Model , we see that the TTCN-3 is structured in two parts: the TE proper and an NAS emulation part, also implemented in TTCN-3 ; so SA has to cover the lower parts of the figure (everything except the upper two yellow blocks).
The port you asked about
type port NBIOT_SRB_PORT message { /* NBIOT PTC: Port for Sending/Receiving data on SRBs */
out NB_SRB_COMMON_REQ;
in NB_SRB_COMMON_IND;
};
is an internal port connecting TE proper with the NAS emulation and as such , needs no further code; just mark it with extension internal, and the rest will be handled by Titan in the background.
Only ports that connect to the system need external C++ code;
so instead of the afore-mentioned NBIOT_SRB_PORT you will need the
NASEMU_NBIOT_SYSTEM_PORT :
type port NASEMU_NBIOT_SYSTEM_PORT message { /* NASEMU PTC: Port for Sending/Receiving data to/from the SYSTEM Interface */
out NBIOT_RRC_PDU_REQ;
in NBIOT_RRC_PDU_IND;
};
where for example
type record NBIOT_RRC_PDU_REQ {
NB_ReqAspCommonPart_Type Common, /* CellId : identifier of the cell
* RoutingInfo : SRB0, SRB1, SRB1bis
* TimingInfo : Now in normal cases;
* For latency tests TimingInfo can be set to the SFN/subframe in which the RRC messages shall be sent out
* NOTE 1: if the RRC PDU is too long to be sent in one TTI the TimingInfo corresponds to the first TTI
* NOTE 2: the TimingInfo is not changed by the NAS Emu (i.e. the timing info as coming from the test case (SRB_COMMON_REQ) is handed through by the NAS Emu)
* ControlInfo
* CnfFlag:=false;
* FollowOnFlag
* true: Indicates that the message(s) to be sent on the same TTI will follow
* NOTE 1: If the TimingInfo is not the same for messages to be sent on the same TTI, the SS shall produce an error
* NOTE 2: the follow on flag applies only for messages of the same SRB
* false: Indicates that no more message(s) will follow */
NB_RRC_MSG_Request_Type RrcPdu
};
:
:
type union NB_RRC_MSG_Request_Type { /* DL RRC PDU on CCCH or DCCH */
DL_CCCH_Message_NB Ccch,
DL_DCCH_Message_NB Dcch
};
:
:
The above figure also tells us that you will also need an NB_SYSIND_PORT ( NB_SYSTEM_IND) , an NB_SYSTEM_PORT (NB_SYSTEM_CTRL_REQ, NB_SYSTEM_CTRL_CNF), and an L2_DATA_PORT(NB_L2_DATA_REQ, NB_L2_DATA_IND).
Your system adapter will have have to terminate all the above four system ports and process the messages appropriately;
but again the exact code of these test ports depends on the architecture and code of the system adaptor.
For some implementation examples please look into the code of the test ports
we have published. However, the greater part of the effort here is the SA.
I hope this makes it clearer
Best regards
Elemer
|
|
|
| Re: Porting 3gpp test suites to Titan [message #1831479 is a reply to message #1831431] |
Thu, 20 August 2020 13:14 |
Mujeeb Ur Rehman
Messages: 16
Registered: January 2020 |
Junior Member |
|
|
Thanks Elemer,
That's a very healthy and clear explanation of over all system. In fact i was searching for this Ref document from 3GPP. Kindly can you give me a ref example of a System or SRB port for sending and receiving a message? please also tell me can i add breakpoint in the compiled CPP files during execution using command line debugger as i want to check the REQ and CNF parameters of the PORT?
Regards,
Mujeeb
[Updated on: Fri, 21 August 2020 06:30] Report message to a moderator |
|
|
| Re: Porting 3gpp test suites to Titan [message #1831504 is a reply to message #1831479] |
Fri, 21 August 2020 07:41 |
|
Hi Mujeeb,
I'm sorry, I don't have a live example of SRB port to offer; my best advice is to look into the already published test ports and figure out from there.
Also, please don't forget that besides the SA, you will have to add codecs to the project; you can either use your own external codecs or use
codecs generated by Titan, for which you will need to decorate the TTCN-3 code with encoding instructions (see my previous posts about codecs, especially the RAW encoder);
whichever path you take , it will mean a serious amount of work.
The Titan debugger works for the TTCN-3 code only; for ports, written in C++, you will have to use appropriate debugging means,
but a print statement added to the port code should probably be sufficient.
Best regards
Elemer
|
|
| | |
Goto Forum:
Current Time: Sat Dec 21 17:11:21 GMT 2024
Powered by FUDForum. Page generated in 0.03837 seconds |
] 
Search
Help
Register
Login
Home
