AS-11 DPP Format Conformance Testing
DPP conformance testing currently focuses on testing product conformance to the UK DPP HD Shim v1.1 of the AMWA AS-11 Specification. Note that AMWA also now hosts the Conformance Rules – a new representation of the UK DPP Shims specified in AS-11 that aim to present the content of this specification in a clearer and more flexible form. The Conformance Rules are used as the final point of reference in cases of discrepancy with the AS-11 specification or ambiguity in the specification.
Categories of Product Functionality
The DPP has defined a number of categories of functionality for which the DPP lab currently offers file format conformance tests. These include: File Writers, File Readers (Players), and File Readers (Transcoders), File Readers (Analysers), File Readers (Specific categories such as Video only or Audio only. Eg: PSE, or loudness monitoring devices) and File Analysers (MXF format and DM validation). Only those products with principal functionality in at least one of these categories are currently being tested. A product must ‘Pass’ or ‘Pass with Conditions’ the tests defined for all categories of functionality offered by the product in order to apply to the AMWA Certification Authority for Certification.
|Products that write AS-11 UK DPP HD files. Tests are carried out to determine whether a device is capable of writing files that conform to the AMWA AS-11 UK DPP HD Shim v1.1.|
File Readers (Players)
|Products that have the ability to read AS-11 DPP HD files and then play the contents of the file to a video and audio monitor. These devices may additionally include the ability to display Timecode, Descriptive Metadata and Programme Parting / Segmentation information. It is not a requirement that products should have all possible functionality. Products are only tested for the features that they have.|
File Readers (Transcoders)
|Products that have the ability to read AS-11 DPP HD files and then transcode the contents to a different format. Transcoded output files are then tested following the Player testing procedure.|
|File Readers (Analysers)||Products that have the ability to read AS-11 DPP HD files and then analyse the MXF layer and Descriptive Metadata, but not necessarily decode the essence.
File Readers (Specific Devices)
|Products that have the ability to read AS-11 DPP HD files and then utilise, analyse or measure video or audio parameters of the Essence, or Descriptive Metadata. (Products may include PSE and loudness monitoring devices, etc)|
|Analysers||Products that have the ability to read AS-11 DPP HD files and then analyse the MXF layer and the Descriptive Metadata against the test criteria for Analysers. (Products may include AQC devices and MXF analyser devices, etc)|
Product Test Reporting
Product Test Reports are issued to manufacturers whose products have successfully been tested. The manufacturer may then submit the report to the AMWA Certification Authority in order to apply for Certification. Test Reports will be available on the AMWA website for all Certified Products and describe the product tested, the testing performed, and the test results.
The Test Reports reference any ‘Test Artefacts’ that were used in the testing. These are listed below:
- AS-11 DPP HD File Conformance Test Suite – F1.x This identifies the tests carried out on AS-11 DPP outputs of the product and describes the file conformance tests used.
- AS-11 DPP HD Reader Tests – R1.x This identifies the File Reader Test procedure (Players and Transcoders), including the list of tests carried out by the OEM on their own product, with the results to be noted.
- AS-11 DPP HD – Reader Tests for Analysers – R2.x This identifies the File Reader Test procedure for Analysers, including the list of tests carried out by the OEM on their own product, with the results to be noted.
- AS-11 DPP HD Reader Tests – R3.x This identifies the File Reader Test procedure for Specific Devices, such as PSE and loudness monitoring, etc, and includes the list of tests carried out by the OEM on their own product, with the results to be noted.
- AS-11 DPP HD Reader Test Files. These AS-11 DPP HD files are used during the testing of File Readers (Players, Transcoders, Specific Products and Analysers).
- AS-11 UK DPP HD – P1 – MXF Tests for Analysers – A1.x (MXF) This identifies the MXF test procedure for Analysers, and lists the MXF features which must be tested by an AS-11 UK DPP test plan, or template, used with an AQC or MXF analyser product. The OEM completed form is assessed as part of the analyser test procedure. If the completed form with test methods is passed by the DPP Compliance Programme then the OEM will be provided with the Part 1 (MXF) test set of files. These should all be tested in the product and the results recorded using the Part 3 procedure.
- AS-11 UK DPP HD – P2 – DM Tests for Analysers – A2.x (DM) This identifies the Descriptive Metadata test procedure for Analysers, and lists the DM features which must be tested by an AS-11 UK DPP test plan, or template, used with an AQC or MXF analyser product. The OEM tests their product with the Part 2 (DM) set of test files and the results are recorded using the Part 3 procedure.
- AS-11 UK DPP HD – P3 – Analyser Test Files – A3.x (FILES) This provides the method to be followed by an Analyser OEM to test their product with the full set of Analyser Test Files (MXF and Descriptive Metadata), and how to record the results.
To access our guides & reports you will need to become a member and login to the membership area.
File Writers – Test Procedure
Stage 1: Once signed up to the DPP Compliance Programme, the manufacturer should send some representative AS-11 DPP HD file samples to the DPP lab to be tested. A File Conformance Test Report is produced which shows how they performed against the conformance criteria. Individual tests may have one of four outcomes: PASS, WARNING, PASS with CONDITIONS, and FAIL. Some tests may just have PASS or FAIL. The overall result takes the individual “worse case” result. WARNINGs are for information and are ignored for overall results. If the initial files tested are a ‘Fail’ then new files will need to be submitted once the product has been updated with a fix for the issue. Once the files are a ‘Pass’, or ‘Pass with Conditions’ then the manufacturer can move to Stage 2 and formally request that the lab test the product at Certification Level.
Stage 2: The manufacturer will need to provide the lab with additional information about the product’s functionality and operation using the ‘Initial OEM Product Submission Form’. The Lab, in discussion with the manufacturer, will then agree the method(s) by which the product being tested will create files for Certification Level Testing, , and this is detailed in a ‘Writer Submission form’.
Once stage 2 testing has been completed the Test Report (showing ‘Pass’ or ‘Pass with Conditions’) is issued to the manufacturer. They can then go ahead and apply for Certification from the AMWA Certification Authority.
Please note: If the device also includes ‘Reader’ functionality then this will also require a ‘Pass’ in order for the Test Report to be issued.
File Readers (Players and Transcoders) – Test Procedure
File Reader testing (Players and Transcoders) is primarily ‘self-serve’. The test procedure may be carried out by the manufacturer at any time. It principally involves downloading the set of AS-11 UK DPP HD Reader test files and asking the product to read each one, and the manufacturer recording the results.
The details are provided in the relevant Reader test form, and the OEM reports the results in the form:
AS-11 DPP HD Reader Tests – R1.x for Players and Transcoders,
AS-11 DPP HD Reader Tests – R2.x for Analysers, and
AS-11 DPP HD Reader Tests – R3.x for Specific Devices.
A declaration form is to be completed and the results returned to the DPP Lab. The results are assessed by The DPP Lab against set criteria which include checks for required functionality. If they are a ‘Pass’ a Test Report is issued to the OEM. The manufacturer can then go ahead and apply for Certification from the AMWA Certification Authority.
Please note: If the device also includes ‘Writer’ functionality then this will also require a ‘Pass’ or ‘Pass with Conditions’, in order for the Test Report to be issued.
File Analysers – Test Procedure
This is concerned with devices whose primary function is File Format Analysis of AS-11 DPP HD files.
This functionality, defined in a specific Test Plan or Test Profile, is tested and assessed as part of the DPP Compliance Programme. This includes tests that the Analyser should be carrying out in order to meet the required Certification Level criteria. There are three parts as follows:
PART 1. MXF Format Tests for File Analysers
PART 2. Descriptive Metadata (DM) Validation Tests
PART 3. File Testing and Reporting (for PART 1 & PART 2)
The Analyser test procedure (Parts 1, 2 and 3) may be carried out by the OEM at any time by following the instructions detailed in the documents. The range of included tests and capabilities is then assessed by the DPP Compliance programme against the Certification Level criteria. If these criteria are met then the Analyser will Pass.
File Reader functionality should be tested and recorded separately, following the “File Reader Tests for Analysers” instructions. This may include other capabilities if present, such as player or transcoder. If the device also writes AS-11 DPP HD files then you must complete Writer tests, and ‘Pass’ or ‘Pass with Conditions’, in order for a device to qualify for Certification.
For all enquiries about DPP membership in order to access the Compliance Programme please contact firstname.lastname@example.org
For general enquiries about the Compliance Programme please contact email@example.com
For further information about certification please contact the AMWA Certification Authority