Configuration Management for Sphinx-4


Managing the Sphinx Configuration

The Sphinx-4 configuration manager system has two primary purposes:

The Configuration File

The configuration of a particular Sphinx-4 system is determined by a configuration file. This configuration file defines the following:
Let's take a look at a simple configuration file:

<config>
<component name="mySampleComponent" type="edu.cmu.sphinx.sample.MyComponent"/>
</config>
 
Some things to note about this configuration file:

Defining components

Now lets look at a somewhat more complex configuration file:
<config>
<component name="mySampleComponent" type="edu.cmu.sphinx.sample.MyComponent"/>
<component name="anotherComponent" type="edu.cmu.sphinx.sample.MyComponent"/>
<component name="aDifferentComponent" type="edu.cmu.sphinx.sample.YourComponent"/>
</config>
This configuration file defines three components, two of the type MyComponent , and one with the type YourComponent. The two components with the same types will result in two different instances of that component being created.

The data types  MyComponent and YourComponent are, of course, fictional types.   Now lets look at a section from a real configuration file.
<config>

<component name="dct" type="edu.cmu.sphinx.frontend.transform.DiscreteCosineTransform"/> <component name="batchCMN" type="edu.cmu.sphinx.frontend.feature.BatchCMN"/> <component name="liveCMN" type="edu.cmu.sphinx.frontend.feature.LiveCMN"/> <component name="featureExtraction" type="edu.cmu.sphinx.frontend.feature.DeltasFeatureExtractor"/>
</config>

Here we see some of the components used in the front end of Sphinx4

Defining configuration data

So far, we've shown how to define new components in the configuration file. Now lets take a look at how we define the detailed configuration data for a component. 

<config>

<component name="concatDataSource" type="edu.cmu.sphinx.frontend.util.ConcatFileDataSource">
<property name="sampleRate" value="16000"/>
<property name="transcriptFile" value="reference.txt"/>
<property name="silenceFile" value="/lab/speech/sphinx4/data/tidigits/test/raw16k/silence1sec.raw"/>
<property name="bytesPerRead" value="320"/>
<property name="batchFile" value="tidigits.batch"/>
<property name="addRandomSilence" value="true"/>
</component>

</config> >
In Sphinx-4,  we call the configuration data for a component  its properties.  Here, we are defining six properties for the concatDataSource component.  Properties are simple name/value pairs and are set with the <property> statement as shown above. 

The properties that can be defined for a component vary based upon the component type.  The API documentation for a component includes a description of the set of properties, the data type for each property, and the default value for each property.  For example a description of the properties used above can be found on the ConcatFileDataSource page.

If a property is omitted from the configuration file, the component will usually provide a default value for the property.

Configuration data types

Sphinx-4 simple properties can be of the following types:
Here are some examples:

value="'Twas brillig and slithey toves"    
string
value="3.14"
float
value="1E-140"
double
value="16000"
integer
value="false"
boolean
value="beamPruner"
component

In addition to these simple property types, there are two list types:
Lists are defined in a propertylist element. Each item in a list is defined with an item  element.  Here's an example of how to define a property list of strings:

<component name="fileManager" type="edu.cmu.sphinx.sample.FileManager">
<propertylist name="fileNames">
<item>file1.txt</item>
<item>file2.txt</item>
<item>file3.txt</item>
</propertylist>
</component>
Property lists of components are defined similarly:
    <component name="mfcLiveFrontEnd" type="edu.cmu.sphinx.frontend.FrontEnd">
<propertylist name="pipeline">
<item>concatDataSource </item>
<item>speechClassifier </item>
<item>speechMarker </item>
<item>nonSpeechDataFilter </item>
<item>preemphasizer </item>
<item>windower </item>
<item>fft </item>
<item>melFilterBank </item>
<item>dct </item>
<item>liveCMN </item>
<item>featureExtraction </item>
</propertylist>
</component>

Error Checking

When a configuration file is loaded, the configuration manager will check for certain errors and abort the process if an error is detected.  Some of the errors that are detected are:

The Elements

The following table details the elements and attributes of the configuration file

Element
Attributes
Sub-elements
Description
<config>
none
<component>
<property>
<propertylist>
The top level element. It has no attributes. It can have any number of the component, property and propertylist sub-elements.
<component>
name - the component name
type - the component type
<property>
<propertylist>
Defines an instance of a component. This element must always have the name and type attributes.
<property>
name -  the property name
value - the type of the property
None
Used to define a single property of a component or a global system property. This element must always have the name and value attributes.
<propertylist>
name  - the name of the property list
<item>
Used to define a list of strings or components. This element must always have the name element. I can have any number of item sub-elements.
<item>
none
none
contents of this element define  a string or a component name.

Global Properties

You may have noticed that properties can be defined out side of any component, at the configuration level.  These are called global properties. Here's an example of some global properties being defined.

<config>
<property name="absoluteBeam" value="1000"/>
<property name="relativeBeam" value="1E-10"/>
</config>

These global variables can then be used in the property statements inside components.  A variable is referenced using the syntax ${variableName}.  To reference the variables defined in the previous example you would use ${absoluteBeam} and ${relativeBeam}

Here's an example of using global properties in a config file:

<config>
<property name="sampleRate" value="16000"/>

<component name="concatDataSource" type="edu.cmu.sphinx.frontend.util.ConcatFileDataSource">
<property name="sampleRate" value="${sampleRate}/>
</component>

<component name="microphone" type="edu.cmu.sphinx.frontend.util.Microphone">
<property name="sampleRate" value="${sampleRate}/>
</component>

<component name="streamDataSource" type="edu.cmu.sphinx.frontend.util.StreamDataSource">
<property name="sampleRate" value="${sampleRate}/>
</component>

</config> >
In this example we have three components, all of which need to be configured with a sampleRate.  We could explicitly use the value "16000" for each of the sampleRate properties, but if we decided to change the sample rate at a later time, we would have to change it in three places.  Using a global property allows us to have a single point where the sample rate is defined. To change the sample rate, we only have to change the single number. 

Global properties are also useful to highlight important tunable parameters. Often times in large configuration files, important parameters that frequently need to be tuned for best results are hidden deep down in the configuration file.  Using global properties, these important, frequently tuned properties can be highlighted.  Here's an example from the tidigits.config.xml file:

<config>        


<!-- ******************************************************** -->
<!-- frequently tuned properties -->
<!-- ******************************************************** -->

<property name="absoluteBeamWidth" value="-1"/>
<property name="relativeBeamWidth" value="1E-200"/>
<property name="wordInsertionProbability" value="1E-36"/>
<property name="languageWeight" value="8"/>
<property name="silenceInsertionProbability" value="1"/>
<property name="skip" value="0"/>

<!-- ******************************************************** -->
<!-- Components -->
<!-- ******************************************************** -->
<component name="batch" type-"..." >
...
</component>



<!-- more omitted .... -->

Global variables can be substituted for all property value  attributes. They can also be used in propertylist item statements.  For example,
here's a configuration of the FrontEnd pipeline that uses a global property to set which cepstral mean normalizer to use:

<config>
<property name="cmn" value="liveCMN"/>

<component name="mfcFrontEnd" type="edu.cmu.sphinx.frontend.FrontEnd">
<propertylist name="pipeline">
<item>streamDataSource</item>
<item>preemphasizer</item>
<item>windower</item>
<item>fft</item>
<item>melFilterBank</item>
<item>dct</item>
<item>${cmn}/item>
<item>featureExtraction</item>
</propertylist>
</component>
</config>

Note that you can not substitute global variables for component names or types.  Thus, this is illegal:

<config>
<property name="cmn" value="liveCMN"/>
<!-- illegal! -->
<component name="${cmn}" type="edu.cmu.sphinx.frontend.CepstralMeanNormalizer">
</component>
</config>

Setting properties from the Java command line

Sometimes it is desirable to set component properties from the java command line.  This is often done from an ant build.xml file. This allows a single configuration file be used to support multiple tests.  The syntax for setting a component property from the command line is:         componentName[propertyName]=value.

For example to set the sampleRate property for the microphone from the command line, you would invoke Java like this:

java -Dmicrophone[sampleRate]=44100 edu.cmu.sphinx.tools.LiveModeRecognizer tidigits.config.xml tidigits.batch
The syntax for global properties is:      globalProperty=value

Here's an example of setting multiple properties, some global and some component properties, from the command line:

java -Dmicrophone[sampleRate]=441000 -DabsoluteBeamWidth=2000 -DwordInsertionProbability=.01 \
edu.cmu.sphinx.tools.LiveModeRecognizer tidigits.config.xml tidigits.batch
Of course, ant  has its own syntax for setting such things. Here's an example of setting properties from an ant build file:
    <target name="tidigits_wordlist_live" description="Live mode TIDIGITS test.">
<java classpath="${classes_dir}" classname="${live_main}"
<sysproperty key="live[skip]" value="1"/>
<sysproperty key="speedTracker[showResponseTime]" value="true"/>
<sysproperty key="frontend" value="mfcLiveFrontEnd"/>
<arg value="${config}"/>
</java>
</target>

Note that currently, it is not possible to set the value of propertylist properties from the command line.

Debugging your configuration

Here are some tips for developing a configuration and getting it to work
Note that properties that are not explicitly defined in the configuration file or as a system property from the command line are shown as [DEFAULT]. This indicates that the internal default for this value is being used.

Full Example

You can find full examples of Sphinx-4 configuration file in sources. For example, check the file
sphinx4/src/apps/edu/cmu/sphinx/demo/transcriber/config.xml


Copyright 1999-2004 Carnegie Mellon University.
Portions Copyright 2002-2004 Sun Microsystems, Inc.
Portions Copyright 2002-2004 Mitsubishi Electric Research Laboratories.
All Rights Reserved. Usage is subject to license terms