-----------------------
Variable Length Packets
-----------------------


As of TPS Version 5.02a, TPS now provides the capability to
process variable lenth packets

What are variable length packets ?
----------------------------------

Variable length packets are packets which contain the
current packet length information within the current
incoming packet, usually a field within the header.

Because variable length packets carry the length information
within themselves, TPS does not need to know, in advance, how
many bytes to read to process a complete packet before
looking for the start of the next packet in the incoming packet
stream. What TPS does require is the position of the length field
within the packet (its Byteindex), how many bytes long the field is
(the Datatype) and any extra processing (Polynomial calibration)
required to convert the value held in the length field to the
total packet length - this latter polynomial
calibration may simply be trivial, i.e. a multiply by one
if the length field contains the total packet length - usually
it doesn't - see CCSDS/PSS format packets below. Because
the Byteindex, Datatype and Polynomial calibration are
all attributes of a standard view-port, you actually configure
a view-port as for any display view-port except the 
view-port is associated (bound) to the packet format - full details
are given further.

As incoming bytes are received, TPS examines the length field,
and then continues to read bytes up to the total packet
length, before deciding a single packet has been received.
It then interprets any future incoming bytes to be
the synchronisation pattern of the next packet - if not,
it generates a lost sync error 0006. Note, any offset to account
for the synchronisation bytes and all bytes up to and including
the length field are configured in the packet length view-port.

Fixed length packets (the default)..
------------------------------------

TPS comes default configured to read packets of a fixed length
of 128 bytes. Admittedly, this is a completely arbitrary
length and, usually, the first thing the user does is
reconfigure this length and synchronisation bit pattern via
the Transfer, Pkt format menu item. However, whatever the
length value configured, TPS expects all packets to be of
the configured length and will always expect a
synchronisation pattern at byte intervals of the fixed
packet length (default 128 bytes). Because the packet length is
configured once, usually upon installation and rarely ever
after, TPS does not need any length information within the packet
to process the data, it always assumes the packets are of the
same size.

In practice, it is often convenient to make packets of variable
length so they can store more information at busy times, e.g.
high sampling rates, and be sufficiently small at slack periods,
i.e. when there is little data to transfer. The simple way
to cope with the variable packet size is to store the
current packet length information within
the current packet, invariably within the packet header
immediately or soon after the synchronisation pattern.
Note, TPS has no real concept of a header except for the
synchronisation patttern - it is down to you how you
define the header but makes no difference to how TPS processes
the packet - this is not a limitation, TPS simply has no need.

When variable length packets can all be of the same fixed length
----------------------------------------------------------------

Although a variable length packet contains its length information,
the actual length need not change and every packet can be of the
same fixed length. When TPS is configured to handle variable length
packets, it will still process the packets as if variable length,
i.e. it will extract the length value and use that plus an optional
calibration to determine the total packet length.
Alternatively, TPS could be configured to process these packet
as fixed length and ignore the length field altogether.


CCSDS and PSS Standard format Telemetry Packets
-----------------------------------------------

TPS's implementation of variable length packets is such that it
can process many different formats, i.e. the length field can
appear anywhere in a packet and be of 1 or 2 bytes long.
However, the majority of users tend to stick with telemetry
packets defined according to the International
CCSDS 102.0-B-2 'Blue Book'
(CCSDS = Consultative Committee on Space Data Systems).
These packets are also conformant with the
ESA's PSS-04-106 'Packet Telemetry Standard' specification document
which is based upon the CCSDS 102.0-B-2 'Blue Book'.

Telemetry packets defined in the CCSDS and PSS standards documents
are referred to as 'PSS' packets in these notes. They may also
be referred to as 'CCSDS' packets in this document, the terms
PSS and CCSDS are interchangeable, i.e. PSS == CCSDS throughout
 this document.

CCSDS/PSS packets, have a packet length field is defined as
a 16-bit (short integer) starting at byteindex 4, the
m.s.byte is at byte 4 and the l.s.byte is at byteindex 5.

IMPORTANT
Do not confuse CCSDS 'transfer frames' with CCSDS 'TM packets'.
Configuration of TPS to read CCSDS transfer frames is not
the same as the configuration to read CCSDS packets. They
are entirely separate issues - 'packets' are extracted from
'transfer frames', and, when talking about CCSDS/PSS packets,
it is assumed that TPS is reading
packets that have already been stripped from the
transfer frames. (See also the 'CCSDS' Online help topic
for details on transfer frames and packet formats).

Back to TPS handling of variable length packets...
--------------------------------------------------

To extract the length, the packet length field must be displayed
in a view-port on the current page or the system page 9999.
Reading the length from a view-port rather than direct from the packet,
allows the user to pre-process the value held in the packet
length field so as to give the total length.
Normally this invloves adding an offset using a polynomial
calibration.
It is rare that the length field ever contains
the exact total packet length, normally it is the number of
bytes that follow the length field minus one

[this is the case with CCSDS packets].

This is because processing software could directly use this as
a counter, decrementing by one as each successive byte is read
and therefore reaching zero when the last byte in the packet
is read. 

However this is just one method of storing the length and,
to enable TPS to process length fields calibrated differently,
TPS makes no prior assumption about the value stored in the
length field and leaves  the fine detail down to a polynomial
calibration specified within a view-port - TPS only requires
that the final value displayed in the view-port is the total 
packet length.

When reading in a packet, TPS reads the sync bytes and
all successive bytes up to and including the packet length field.
This means, for PSS packets, TPS first reads the entire 
six byte header - the last two bytes being the packet length
field. TPS will extract the length field value and add an
offset, as explained following, to determine the complete
packet length. It will then attempt to read the remaining
bytes in the packet.


How to configure TPS to process variable length packet OVERVIEW
---------------------------------------------------------------

Here is an overview of how to configure TPS to read
variable length packets. Full examples are given following.

There are two main steps...

1) Configure a packet length view-port

2) Bind (associate) this view-port to the packet format


1) Configure a packet length view-port
--------------------------------------

The packet length view-port should be configured to display
the total packet length. Usually,
you must configure your view-port with a polynomial to add
an offset accounting for the number of bytes already read
which are not usually included in the length field.

For PSS packets, this offset is fixed at 6 for the 6-byte
header. However, since the length information is actually
reduced by 1, e.g. it has a value of 127 if 128 bytes follow
the 6-byte header, you must also add an additional 1 to the
6 figure. Thus, the total packet length as displayed in the
view-port has an offset of 7 added to it (poly coefficient
a1=1, a0=7 - see polynomial calibration.

Note, putting the view port on page 9999 ensures it is 
displayed on EVERY page - this configuration can only be done
offline.


2) Bind (associate) this view-port to the packet format

Once a view-port is configured, you must then bind
the view-port to the packet format - this is so that TPS
knows which view-port to lookup the total length.
 - you do this bind operation via the Transfer, Pkt
format menu item.

When selected, the Transfer, Pkt format menu item pops up
a dialog box showing the packet sync and length. by setting
the length to 0, this informs TPS that incoming packets do not
have a predefined length. Immediately a packet length value
of 0 is entered, a list box appears showing selected parameters
on the current page or page 9999. From this list, you select
the view-port pre-configured to display the packet length.

The list only contains those displayed parameters with
view-ports configured as...

A packet header parameter (obviously length field is held
in the packet and not an internal TPS variable).

A numeric (integer or real) datatype, e.g. not %Hc or %c

Not super or sub-com


IMPORTANT - The packet length view-port MUST be properly
configured before selecting it because you will not be able to
edit it thereafter unless you set the packet length back
to a fixed, non zero value (any non-zero length will do)
and then correct it. After correcting, you must then repeat
packet format configuration, i.e. setting the length to 0 and
re-selecting the corrected view-port. TPS does not do this
automatically.

Set the TPS sync to recognise the first
two bytes of the packet (the Application ID) to match those
of the incoming packets. This is because, of course, the
sync pattern is entirely user-dependant and not part
of the PSS packet format - only the length and position
of the sync field is defined, not its actual contents.
To do this, select the Transfer, Pkt format menu item and
modify the sync accordingly.

DO NOT change the packet length from zero,
this setting effectively tells TPS that the incoming
data is variable length.

Because view-ports are page specific, if you turn to
a page that doesn't have a packet length view-port
configured, TPS will not know the current packet length.
This has the advantage that different
pages can be configured to read packets with different
formats. In reality, this is rarely (never) the case and
so, to avoid having to manually configure the same
packet length view-port on every page, you can add
the view-port to system page 9999 (done offline) which
will then appear on every page.

[valueport]
	databasekey = "SAMPLE_Total_Packet_"
	view = packetheader
	byteindex = 4
	datatype = "%hu"
	row = 1
	column = 1
	mnemonic = "Total_Packet_Length"
	displayrate = "F01ST01"
	backgroundcolor = blue
	bordercolor = white
	titlecolor = white
	valuecolor = yellow
	border = off
	varpktlen = on
	[calibration]
		function = polynomial
		[polynomial]
			function = integerbyteswap
			a4 = 0
			a3 = 0
			a2 = 0
			a1 = 1
			a0 = 7

This was generated as follows (overview only):

  ONLINE: generation of a view-port 'Total_Packet_length'
  on a test page (arbitrary free page).
  The 'Total_Packet_length' view-port displays the total
  PSS packet length  (see line a0 = 7 which adds the 7-byte
  offset as mentioned above).

  In fact, this view-port is a modified version of the view-port
  'source_Packet_Length' found on the CCSDS packet page 34
  on the Samples disk.

  Save the test page.

  OFFLINE: using a text editor, copy the view-port
  specification to PAGE9999.TXT and then delete the text
  from the test page.

  OFFLINE: recompile the test page and page9999 to generate
  PAGE9999.BIN - which is read by TPS when first starting TPS.

For full details on offline page configuration, see
the manual 'PGCKW and Offline Configuration' - supplied
in Word 6 format as PGCKW.DOC, or MS Write format PGCKW.WRI in
the Docs sub-folder.


Once the sync (Application Id) is configured, TPS is ready
to read variable length packets - simply replay the
data as for standard fixed length packets.

Test on fixed length packets

When using any TPS to read variable length packets,
the first thing we suggest is that you actually
read packets that have a fixed length but, like
variable length packets, contain the length
embedded in the packet. As they are fixed length,
the length field in the packet header will not vary. This
will enable you to ensure your packet length view-port is
correctly configured and that TPS can read data properly,
i.e. not lose sync, when processing variable length
packets.


How do I know the current configuration state with regard to
------------------------------------------------------------
variable or fixed packet length handling?
-----------------------------------------

The current packet format configuration information can be found
from several sources:

1) Select the Transfer, Pkt format menu item

	This shows the packet sync and length, if the length
	is zero, then TPS is processing variable length
	packets otherwise, when non-zero, TPS is
	processing fixed length packets.

2) Displayed on every page

	Top left, blue-background view-port 'Total_Packet_Length'

	This will show '7' when first starting TPS.
	After a PSS packet is replayed, it will
	show the packet length as extracted
	from the current packet.

3) Page 10

	Above hex pkt dump on left are two view-ports:

	PKTSYNC	    packet sync (PSS app. Id)
	PKTLENGTH = 0 variable length pkts
		  = non-zero fixed length pkts (shows
		    fixed length pkt size).

	Var_Pkt_Length

	See the grey background block, top right-hand side,
	headed 	'General' to find this parameter - it is at
	the foot of the block. The grey block shows various
	TPS configuration states including the variable
	packet length ON/OFF state.
	When using fixed length packets, i.e. a non-zero length
	value in the packet format dialog box, 'Var_pkt_length'
	shows the OFF state. It shows ON when the packet length
	is variable, i.e. the packet format dialog box
	shows a packet length of 0. 


4) See Page 34 on the Samples disk for CCSDS packet header
displays - all bit fields are extracted.

5) Select Options, Info menu item and press the 'Var. Pkt Length'
button. When using variable length pkts, this will tell you which
mnemonic (parameter, view-port) is being used to retrieve the
length information.


To revert to fixed length packets
---------------------------------
If you wish to receive fixed length packets, i.e the packet
length information is not contained in the packet or you
just want TPS to ignore it:

1. Select the Transfer, Pkt format menu item and enter the
fixed length, in bytes, in the Length box. Also enter the new
sync pattern if required.

2. Press OK to accept the changes.

TPS is then ready to replay fixed length packets.

If you wish to receive packets in the original PSS format

1. Select the Transfer, Pkt format menu item and enter zero
in the length field. TPS will then display a parameter
mnemonic list box.

2. Select the 'Total_Packet_Length' from the list box.

3. Set the sync as desired to the application id.

4. Press OK to accept the changes.

TPS is now ready to replay PSS conformant variable length
packets.

If you wish to receive variable length packets NOT
--------------------------------------------------
in the PSS format.
------------------

To do this, you have to reconfigure the Page 9999 view-port
'Total_Packet_length'. This is, however, pre-configured on
the page 9999 which cannot be directly modified online as
for a normal view-port. However, you can make a copy offline
using a text editor (offline) and then edit the copy online
as for a standard view-port. Details follow...

Other than being displayed on page9999, "Total_Packet_Length"
is a standard view-port that displays the PSS packet length
field with an offset of 7 added to display the total packet
length.

The 'Total_Packet_Length' view-port configuration,
as found in the page 9999 file PAGE9999.TXT is shown below.

[valueport]
	databasekey = "Total_Packet_Length"
	view = packetheader
	byteindex = 4
	datatype = "%hu"
	row = 1
	column = 1
	mnemonic = "Total_Packet_Length"
	displayrate = "F01ST01"
	backgroundcolor = blue
	bordercolor = white
	titlecolor = white
	valuecolor = yellow
	border = off
	varpktlen = on
	[calibration]
		function = polynomial
		[polynomial]
			function = integerbyteswap
			a4 = 0
			a3 = 0
			a2 = 0
			a1 = 1
			a0 = 7

To edit this view-port:

Do all the below operations in the installed TPS root
directory.

Make a backup copy of PAGE9999.TXT just in case.

Start a text editor, e.g. Windows Notepad, and
edit the file PAGE9999.TXT.

Cut the text block from PAGE9999.txt and save
the PAGE9999.TXT file with the view-port text block
removed, (hence a 'cut' operation and not a 'copy').
Then whilst still in the editor, paste the text block
to a new page file. Alternatively, you can paste it at the
end of one of your existing pages, for example, if you
have a display page 55, then paste it to the end of the
PAGE0055.TXT file. It is sometimes better to use a new
page just for starters. For a new page, say 99,
when you have pasted the text block into the new file,
save the page using the name PAGE0099.TXT.

Note, if you use another free page, always remember
to use a filename of the form PAGEnnnn.TXT where nnnn
is an unused 4-digit page number - padded with leading
zeros to make 4 digits.

Quit the editor and, offline, recompile both page 9999
and your new page nnnn using PGCKW - simply run PGCKW from
the command line or the TPS menu bar.

For full details on offline page configuration, see
the manual 'PGCKW and Offline Configuration' - supplied
in Word 6 format as PGCKW.DOC, or MS Write format PGCKW.WRI in
the Docs sub-folder.

Once PGCKW is started, select File, Compile, from the PGCKW
menu and select the PAGE9999.TXT file to compile. Repeat this
for your page. PGCKW will display a message when completed.

So far, all you have done is made a new page with a copy
of the original packet length view-port. Page 9999 no longer
has this view-port so, when you next start TPS, it will
revert back to its default 128 byte fixed packet length.

Start TPS and change to your new page showing the packet
length view-port. Although this page has the packet
length view-port you cannot, at this stage, read variable
length packets because TPS has not been re-configured
to recognise this view-port as that which gives it its
packet length information. As far as TPS is concerned,
this view-port is the same as any other. Before you
reconfigure TPS to recognise this view-port, you must
configure this view-port to display the contents of the 
packet length field with any polynomial corrections
to ensure the value displayed matches the total packet
length.

Invariably this means changing the Byteindex, Dataype and,
probably, the polynomial coefficients (see a0).

If the packet length field does not contain the total number
of bytes in the packet but rather, say, the total minus one 
or, in the case of the PSS packets, the total minus one
and minus the header length (six), then usually you
will have to adjust the polynomial coefficent a0 to add
in the extra bytes to ensure the view-port shows the total
length. Because you cannot actually read the packets
before this view-port is configured, it may be difficult
to get the view-port to display the right value without
any data. If this is the case, we suggest you read in some
packets (or just one packet), where the length field is present
but has a fixed value, i.e. as for fixed length packets.
To read in one packet, just put TPS into step mode
before replaying the data  - press ENTER once to replay
a single packet.

Once you are happy the packet length view-port diplays the
correct value, i.e. the total packet length, then
save the page online.

At this stage, you could do a full test of your view-port
configuration by remaining on the test page (do not change to
any other page) and replaying several packets.

Once you are fully satsified the packet length view-port
is correct, you are ready to transfer the view-port back to
page 9999 so that TPS will then replay the variable length
packets regardless of the current page displayed. This is
similar to the cut and paste operation described above. Once
again, because you are cutting the view-port from
your user page to page 9999, you will have to do the
operation offline with a text editor - system pages, i.e.
page 9999, can only be edited offline.

Start a text editor, e.g. Windows Notepad, and
edit your test page file PAGEnnnn.TXT.

Cut the packet length view-port configuration block 
(starts with [valueport] and ends
on the line immediately preceeding the next [valueport]
or end of the file) and then paste this to PAGE9999.TXT
at the end of the page file.

Save the newly modified page file PAGE9999.TXT and then
delete your test page.

Recompile page 9999 using PGCKW as for the instructions
previously.

Now start TPS, the final operation you have to do
is to configure TPS to recognise your new packet length
view-port as that which contains the data length. To do
this, select the Transfer, Pkt format menu item and
enter in a value of 0 for the packet length. TPS will now
display a list of mnemonics for all numeric view-ports
on display which can be used to display the packet length
(only those of real or integer displaytype).
select the packet length view-port. Once selected,
check the sync pattern is correct and then press
OK to accept all the changes.

TPS is now configured to replay variable length packets.
If this is OK, i.e. no unexpected lost sync errors, you
can turn to any of your other display pages and TPS
should remain configured to continue reading incoming
variable length packets. Whatever page you change to,
you should see your packet length view-port (unless it
is hidden by another view-port).

Finally, save the system settings, either now or when you
exit TPS. To save the system settings at any time, select
the Options, System, Save Settings menu item.

This completes the re-configuration process.


Samples, Using Variable length packets..
Pre-configured version on request...

See Page 34 on the Samples disk for CCSDS packet header
displays - all bit fields extracted.

Displayed in the top left of this sample page, 
is a blue-background view-port with mnemonic
'SAMPLE_Total_Packet_Length'.
This will read '7' when first starting TPS. After a PSS
packet is replayed, it will show the packet length as
extracted from the current packet.

[valueport]
	databasekey = "SAMPLE_Total_Packet_"
	view = packetheader
	byteindex = 4
	datatype = "%hu"
	row = 1
	column = 1
	mnemonic = "Total_Packet_Length"
	displayrate = "F01ST01"
	backgroundcolor = blue
	bordercolor = white
	titlecolor = white
	valuecolor = yellow
	border = off
	varpktlen = on
	[calibration]
		function = polynomial
		[polynomial]
			function = integerbyteswap
			a4 = 0
			a3 = 0
			a2 = 0
			a1 = 1
			a0 = 7

Documentation Availability...
Online online
