[comment {-*- tcl -*- doctools manpage}]
[manpage_begin doctools::idx::import n 0.1]
[copyright {2009 Andreas Kupries <andreas_kupries@users.sourceforge.net>}]
[moddesc   {Documentation tools}]
[titledesc {Importing keyword indices}]
[category  {Documentation tools}]
[require doctools::idx::import [opt 0.1]]
[require Tcl 8.4]
[require doctools::config]
[require doctools::idx::structure]
[require snit]
[require pluginmgr]
[keywords {keyword index} index reference documentation manpage url]
[keywords markup conversion import parsing plugin]
[keywords json docidx]
[description]

This package provides a class to manage the plugins for the import of
keyword indices from other formats, i.e. their conversion from, for
example [term docidx], [term json], etc.

[para]

This is one of the three public pillars the management of keyword
indices resides on. The other two pillars are 

[list_begin enum]
[enum] [manpage {Exporting keyword indices}], and
[enum] [manpage {Holding keyword indices}]
[list_end]

[para]

For information about the [sectref Concepts] of keyword indices, and
their parts, see the same-named section.

For information about the data structure which is the major output of
the manager objects provided by this package see the section
[sectref {Keyword index serialization format}].

[para]

The plugin system of our class is based on the package
[package pluginmgr], and configured to look for plugins using

[list_begin enum]
[enum] the environment variable [var DOCTOOLS_IDX_IMPORT_PLUGINS],
[enum] the environment variable [var DOCTOOLS_IDX_PLUGINS],
[enum] the environment variable [var DOCTOOLS_PLUGINS],
[enum] the path [file {~/.doctools/idx/import/plugin}]
[enum] the path [file {~/.doctools/idx/plugin}]
[enum] the path [file {~/.doctools/plugin}]
[enum] the path [file {~/.doctools/idx/import/plugins}]
[enum] the path [file {~/.doctools/idx/plugins}]
[enum] the path [file {~/.doctools/plugins}]
[enum] the registry entry "HKEY_CURRENT_USER\SOFTWARE\DOCTOOLS\IDX\IMPORT\PLUGINS"
[enum] the registry entry "HKEY_CURRENT_USER\SOFTWARE\DOCTOOLS\IDX\PLUGINS"
[enum] the registry entry "HKEY_CURRENT_USER\SOFTWARE\DOCTOOLS\PLUGINS"
[list_end]

The last three are used only when the package is run on a machine
using Windows(tm) operating system.

[para]

The whole system is delivered with two predefined import plugins,
namely

[list_begin definitions]
[def docidx]  See [manpage {docidx import plugin}] for details.
[def json]    See [manpage {json import plugin}] for details.
[list_end]

[para]

Readers wishing to write their own import plugin for some format, i.e.
[term {plugin writer}]s reading and understanding the section
containing the [sectref {Import plugin API v2 reference}] is an
absolute necessity, as it specifies the interaction between this
package and its plugins in detail.


[section Concepts] [include include/concept.inc]


[section API]
[subsection {Package commands}]

[list_begin definitions]

[call [cmd ::doctools::idx::import] [arg objectName]]

This command creates a new import manager object with an associated
Tcl command whose name is [arg objectName]. This [term object] command
is explained in full detail in the sections [sectref {Object command}]
and [sectref {Object methods}]. The object command will be created
under the current namespace if the [arg objectName] is not fully
qualified, and in the specified namespace otherwise.

[list_end]


[subsection {Object command}]

All objects created by the [cmd ::doctools::idx::import] command have
the following general form:

[list_begin definitions]

[call [cmd objectName] [method method] [opt [arg "arg arg ..."]]]

The method [method method] and its [arg arg]'uments determine the
exact behavior of the command.

See section [sectref {Object methods}] for the detailed
specifications.

[list_end]


[subsection {Object methods}]

[list_begin definitions]

[call [arg objectName] [method destroy]]

This method destroys the object it is invoked for.


[call [arg objectName] [method {import text}] [arg text] [opt [arg format]]]

This method takes the [arg text] and converts it from the specified
[arg format] to the canonical serialization of a keyword index using
the import plugin for the format. An error is thrown if no plugin
could be found for the format.

The serialization generated by the conversion process is returned as
the result of this method.

[para]

If no format is specified the method defaults to [const docidx].

[para]

The specification of what a [term canonical] serialization is can be
found in the section [sectref {Keyword index serialization format}].

[para]

The plugin has to conform to the interface specified in section
[sectref {Import plugin API v2 reference}].


[call [arg objectName] [method {import file}] [arg path] [opt [arg format]]]

This method is a convenient wrapper around the [method {import text}]
method described by the previous item.

It reads the contents of the specified file into memory, feeds the
result into [method {import text}] and returns the resulting
serialization as its own result.


[call [arg objectName] [method {import object text}] [arg object] \
	[arg text] [opt [arg format]]]

This method is a convenient wrapper around the [method {import text}]
method described by the previous item.

It expects that [arg object] is an object command supporting a
[method deserialize] method expecting the canonical serialization of a
keyword index.

It imports the text using [method {import text}] and then feeds the
resulting serialization into the [arg object] via [method deserialize].

This method returns the empty string as it result.


[call [arg objectName] [method {import object file}] [arg object] \
	[arg path] [opt [arg format]]]

This method behaves like [method {import object text}], except that it
reads the text to convert from the specified file instead of being
given it as argument.


[call [arg objectName] [method {config names}]]

This method returns a list containing the names of all configuration
variables currently known to the object.


[call [arg objectName] [method {config get}]]

This method returns a dictionary containing the names and values of
all configuration variables currently known to the object.


[call [arg objectName] [method {config set}] [arg name] [opt [arg value]]]

This method sets the configuration variable [arg name] to the
specified [arg value] and returns the new value of the variable.

[para]

If no value is specified it simply returns the current value, without
changing it.

[para]

Note that while the user can set the predefined configuration
variables [const user] and [const format] doing so will have no
effect, these values will be internally overriden when invoking an
import plugin.


[call [arg objectName] [method {config unset}] [arg pattern]...]

This method unsets all configuration variables matching the specified
glob [arg pattern]s. If no pattern is specified it will unset all
currently defined configuration variables.


[call [arg objectName] [method includes]]

This method returns a list containing the currently specified paths to
use to search for include files when processing input.

The order of paths in the list corresponds to the order in which they
are used, from first to last, and also corresponds to the order in
which they were added to the object.


[call [arg objectName] [method {include add}] [arg path]]

This methods adds the specified [arg path] to the list of paths to use
to search for include files when processing input. The path is added
to the end of the list, causing it to be searched after all previously
added paths. The result of the command is the empty string.

[para]

The method does nothing if the path is already known.


[call [arg objectName] [method {include remove}] [arg path]]

This methods removes the specified [arg path] from the list of paths
to use to search for include files when processing input. The result
of the command is the empty string.

[para]

The method does nothing if the path is not known.


[call [arg objectName] [method {include clear}]]

This method clears the list of paths to use to search for include
files when processing input. The result of the command is the empty
string.

[list_end]


[section {Import plugin API v2 reference}]

Plugins are what this package uses to manage the support for any input
format beyond the [sectref {Keyword index serialization format}]. Here
we specify the API the objects created by this package use to interact
with their plugins.

[para]

A plugin for this package has to follow the rules listed below:

[list_begin enumerated]

[enum] A plugin is a package.

[enum] The name of a plugin package has the form

		doctools::idx::import::[var FOO],

       where [var FOO] is the name of the format the plugin will
       generate output for. This name is also the argument to provide
       to the various [method import] methods of import manager
       objects to get a string encoding a keyword index in that
       format.

[enum] The plugin can expect that the package
       [package doctools::idx::export::plugin] is present, as
       indicator that it was invoked from a genuine plugin manager.

[enum] The plugin can expect that a command named [cmd IncludeFile] is
       present, with the signature

[list_begin definitions]
[call [cmd IncludeFile] [arg currentfile] [arg path]]

This command has to be invoked by the plugin when it has to process an
included file, if the format has the concept of such. An example of
such a format would be [term docidx].

[para]
The plugin has to supply the following arguments

[list_begin arguments]
[arg_def string currentfile]
The path of the file it is currently processing. This may be the empty
string if no such is known.

[arg_def string path]
The path of the include file as specified in the include directive
being processed.

[list_end]

The result of the command will be a 5-element list containing

[list_begin enum]

[enum] A boolean flag indicating the success ([const True]) or failure
       ([const False]) of the operation.

[enum] In case of success the contents of the included file, and the
       empty string otherwise.

[enum] The resolved, i.e. absolute path of the included file, if
       possible, or the unchanged [arg path] argument. This is for
       display in an error message, or as the [arg currentfile]
       argument of another call to [cmd IncludeFile] should this file
       contain more files.

[enum] In case of success an empty string, and for failure a code
       indicating the reason for it, one of

[list_begin definitions]
[def notfound] The specified file could not be found.
[def notread]  The specified file was found, but not be read into memory.
[list_end][comment {-- include error codes --}]

[enum] An empty string in case of success of a [const notfound]
       failure, and an additional error message describing the reason
       for a [const notread] error in more detail.

[list_end][comment {-- result list elements --}]
[list_end][comment {-- include-file signature --}]

[enum] A plugin has to provide one command, with the signature shown
       below.

[list_begin definitions]
[call [cmd import] [arg text] [arg configuration]]

Whenever an import manager of [package doctools::idx] has to parse
input for an index it will invoke this command.

[list_begin arguments]

[arg_def string text]

This argument will contain the text encoding the index per the format
the plugin is for.

[arg_def dictionary configuration]

This argument will contain the current configuration to apply to the
parsing, as a dictionary mapping from variable names to values.

[para]

The following configuration variables have a predefined meaning all
plugins have to obey, although they can ignore this information at
their discretion. Any other other configuration variables recognized
by a plugin will be described in the manpage for that plugin.

[list_begin definitions]

[def user] This variable is expected to contain the name of the user
           owning the process invoking the plugin.

[def format] This variable is expected to contain the name of the
             format whose plugin is invoked.

[list_end][comment {-- predefined configuration variables --}]
[list_end][comment {-- arguments --}]
[list_end][comment {-- api command signatures --}]

[enum] A single usage cycle of a plugin consists of the invokations of
       the command [cmd import]. This call has to leave the plugin in
       a state where another usage cycle can be run without problems.

[list_end]


[include include/serialization.inc]
[vset CATEGORY doctools]
[include ../doctools2base/include/feedback.inc]
[manpage_end]