Classes for sources of doctests#

This module defines various classes for sources from which doctests originate, such as files, functions or database entries.

AUTHORS:

  • David Roe (2012-03-27) – initial version, based on Robert Bradshaw’s code.

class sage.doctest.sources.DictAsObject(attrs)#

Bases: dict

A simple subclass of dict that inserts the items from the initializing dictionary into attributes.

EXAMPLES:

sage: from sage.doctest.sources import DictAsObject
sage: D = DictAsObject({'a':2})
sage: D.a
2
class sage.doctest.sources.DocTestSource(options)#

Bases: object

This class provides a common base class for different sources of doctests.

INPUT:

file_optional_tags()#

Return the set of tags that should apply to all doctests in this source.

This default implementation just returns the empty set.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import StringDocTestSource, PythonSource
sage: from sage.structure.dynamic_class import dynamic_class
sage: s = "'''\n    sage: 2 + 2\n    4\n'''"
sage: PythonStringSource = dynamic_class('PythonStringSource', (StringDocTestSource, PythonSource))
sage: PSS = PythonStringSource('<runtime>', s, DocTestDefaults(), 'runtime')
sage: PSS.file_optional_tags
set()
class sage.doctest.sources.FileDocTestSource(path, options)#

Bases: DocTestSource

This class creates doctests from a file.

INPUT:

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','sources.py')
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS.basename
'sage.doctest.sources'
basename()#

The basename of this file source, e.g. sage.doctest.sources

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC,'sage','rings','integer.pyx')
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS.basename
'sage.rings.integer'
create_doctests(namespace)#

Return a list of doctests for this file.

INPUT:

OUTPUT:

  • doctests – a list of doctests defined in this file.

  • extras – a dictionary

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','sources.py')
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: doctests, extras = FDS.create_doctests(globals())
sage: len(doctests)
43
sage: extras['tab']
False

We give a self referential example:

sage: doctests[20].name
'sage.doctest.sources.FileDocTestSource.create_doctests'
sage: doctests[20].examples[10].source
'doctests[Integer(20)].examples[Integer(10)].source\n'
file_optional_tags()#

Return the set of tags that should apply to all doctests in this source.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC, 'sage', 'repl', 'user_globals.py')
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: FDS.file_optional_tags
{'sage.modules': None}
in_lib()#

Whether this file is to be treated as a module in a Python package.

Such files aren’t loaded before running tests.

This uses is_package_or_sage_namespace_package_dir() but can be overridden via DocTestDefaults.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC, 'sage', 'rings', 'integer.pyx')
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: FDS.in_lib
True
sage: filename = os.path.join(SAGE_SRC, 'sage', 'doctest', 'tests', 'abort.rst')
sage: FDS = FileDocTestSource(filename, DocTestDefaults())
sage: FDS.in_lib
False

You can override the default:

sage: FDS = FileDocTestSource("hello_world.py",DocTestDefaults())
sage: FDS.in_lib
False
sage: FDS = FileDocTestSource("hello_world.py",DocTestDefaults(force_lib=True))
sage: FDS.in_lib
True
printpath()#

Whether the path is printed absolutely or relatively depends on an option.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.env import SAGE_SRC
sage: import os
sage: root = os.path.realpath(os.path.join(SAGE_SRC,'sage'))
sage: filename = os.path.join(root,'doctest','sources.py')
sage: cwd = os.getcwd()
sage: os.chdir(root)
sage: FDS = FileDocTestSource(filename,DocTestDefaults(randorder=0,abspath=False))
sage: FDS.printpath
'doctest/sources.py'
sage: FDS = FileDocTestSource(filename,DocTestDefaults(randorder=0,abspath=True))
sage: FDS.printpath
'.../sage/doctest/sources.py'
sage: os.chdir(cwd)
class sage.doctest.sources.PythonSource#

Bases: SourceLanguage

This class defines the functions needed for the extraction of doctests from python sources.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','sources.py')
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: type(FDS)
<class 'sage.doctest.sources.PythonFileSource'>
ending_docstring(line)#

Determines whether the input line ends a docstring.

INPUT:

  • line – a string, one line of an input file.

OUTPUT:

  • an object that, when evaluated in a boolean context, gives True or False depending on whether the input line marks the end of a docstring.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.util import NestedName
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','sources.py')
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS._init()
sage: FDS.quotetype = "'''"
sage: FDS.ending_docstring("'''")
<...Match object...>
sage: FDS.ending_docstring('\"\"\"')
start_finish_can_overlap = False#
starting_docstring(line)#

Determines whether the input line starts a docstring.

If the input line does start a docstring (a triple quote), then this function updates self.qualified_name.

INPUT:

  • line – a string, one line of an input file

OUTPUT:

  • either None or a Match object.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.util import NestedName
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','sources.py')
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS._init()
sage: FDS.starting_docstring("r'''")
<...Match object...>
sage: FDS.ending_docstring("'''")
<...Match object...>
sage: FDS.qualified_name = NestedName(FDS.basename)
sage: FDS.starting_docstring("class MyClass():")
sage: FDS.starting_docstring("    def hello_world(self):")
sage: FDS.starting_docstring("        '''")
<...Match object...>
sage: FDS.qualified_name
sage.doctest.sources.MyClass.hello_world
sage: FDS.ending_docstring("    '''")
<...Match object...>
sage: FDS.starting_docstring("class NewClass():")
sage: FDS.starting_docstring("    '''")
<...Match object...>
sage: FDS.ending_docstring("    '''")
<...Match object...>
sage: FDS.qualified_name
sage.doctest.sources.NewClass
sage: FDS.starting_docstring("print(")
sage: FDS.starting_docstring("    '''Not a docstring")
sage: FDS.starting_docstring("    ''')")
sage: FDS.starting_docstring("def foo():")
sage: FDS.starting_docstring("    '''This is a docstring'''")
<...Match object...>
class sage.doctest.sources.RestSource#

Bases: SourceLanguage

This class defines the functions needed for the extraction of doctests from ReST sources.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_doc.rst"
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: type(FDS)
<class 'sage.doctest.sources.RestFileSource'>
ending_docstring(line)#

When the indentation level drops below the initial level the block ends.

INPUT:

  • line – a string, one line of an input file

OUTPUT:

  • a boolean, whether the verbatim block is ending.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_doc.rst"
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS._init()
sage: FDS.starting_docstring("Hello world::")
True
sage: FDS.ending_docstring("    sage: 2 + 2")
False
sage: FDS.ending_docstring("    4")
False
sage: FDS.ending_docstring("We are now done")
True
parse_docstring(docstring, namespace, start)#

Return a list of doctest defined in this docstring.

Code blocks in a REST file can contain python functions with their own docstrings in addition to in-line doctests. We want to include the tests from these inner docstrings, but Python’s doctesting module has a problem if we just pass on the whole block, since it expects to get just a docstring, not the Python code as well.

Our solution is to create a new doctest source from this code block and append the doctests created from that source. We then replace the occurrences of “sage:” and “>>>” occurring inside a triple quote with “safe:” so that the doctest module doesn’t treat them as tests.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.parsing import SageDocTestParser
sage: from sage.doctest.util import NestedName
sage: filename = "sage_doc.rst"
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS.parser = SageDocTestParser(set(['sage']))
sage: FDS.qualified_name = NestedName('sage_doc')
sage: s = "Some text::\n\n    def example_python_function(a, \
....:      b):\n        '''\n        Brief description \
....:      of function.\n\n        EXAMPLES::\n\n            \
....:      sage: test1()\n            sage: test2()\n        \
....:      '''\n        return a + b\n\n    sage: test3()\n\nMore \
....:      ReST documentation."
sage: tests = FDS.parse_docstring(s, {}, 100)
sage: len(tests)
2
sage: for ex in tests[0].examples:
....:     print(ex.sage_source)
test3()
sage: for ex in tests[1].examples:
....:     print(ex.sage_source)
test1()
test2()
sig_on_count() # check sig_on/off pairings (virtual doctest)
start_finish_can_overlap = True#
starting_docstring(line)#

A line ending with a double colon starts a verbatim block in a ReST file, as does a line containing .. CODE-BLOCK:: language.

This function also determines whether the docstring block should be joined with the previous one, or should be skipped.

INPUT:

  • line – a string, one line of an input file

OUTPUT:

  • either None or a Match object.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_doc.rst"
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS._init()
sage: FDS.starting_docstring("Hello world::")
True
sage: FDS.ending_docstring("    sage: 2 + 2")
False
sage: FDS.ending_docstring("    4")
False
sage: FDS.ending_docstring("We are now done")
True
sage: FDS.starting_docstring(".. link")
sage: FDS.starting_docstring("::")
True
sage: FDS.linking
True
class sage.doctest.sources.SourceLanguage#

Bases: object

An abstract class for functions that depend on the programming language of a doctest source.

Currently supported languages include Python, ReST and LaTeX.

parse_docstring(docstring, namespace, start)#

Return a list of doctest defined in this docstring.

This function is called by DocTestSource._process_doc(). The default implementation, defined here, is to use the sage.doctest.parsing.SageDocTestParser attached to this source to get doctests from the docstring.

INPUT:

  • docstring – a string containing documentation and tests.

  • namespace – a dictionary or sage.doctest.util.RecordingDict.

  • start – an integer, one less than the starting line number

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: from sage.doctest.parsing import SageDocTestParser
sage: from sage.doctest.util import NestedName
sage: from sage.env import SAGE_SRC
sage: import os
sage: filename = os.path.join(SAGE_SRC,'sage','doctest','util.py')
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: doctests, _ = FDS.create_doctests({})
sage: for dt in doctests:
....:     FDS.qualified_name = dt.name
....:     dt.examples = dt.examples[:-1] # strip off the sig_on() test
....:     assert(FDS.parse_docstring(dt.docstring,{},dt.lineno-1)[0] == dt)
class sage.doctest.sources.StringDocTestSource(basename, source, options, printpath, lineno_shift=0)#

Bases: DocTestSource

This class creates doctests from a string.

INPUT:

  • basename – string such as ‘sage.doctests.sources’, going into the names of created doctests and examples.

  • source – a string, giving the source code to be parsed for doctests.

  • options – a sage.doctest.control.DocTestDefaults or equivalent.

  • printpath – a string, to be used in place of a filename when doctest failures are displayed.

  • lineno_shift – an integer (default: 0) by which to shift the line numbers of all doctests defined in this string.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import StringDocTestSource, PythonSource
sage: from sage.structure.dynamic_class import dynamic_class
sage: s = "'''\n    sage: 2 + 2\n    4\n'''"
sage: PythonStringSource = dynamic_class('PythonStringSource',(StringDocTestSource, PythonSource))
sage: PSS = PythonStringSource('<runtime>', s, DocTestDefaults(), 'runtime')
sage: dt, extras = PSS.create_doctests({})
sage: len(dt)
1
sage: extras['tab']
[]
sage: extras['line_number']
False

sage: s = "'''\n\tsage: 2 + 2\n\t4\n'''"
sage: PSS = PythonStringSource('<runtime>', s, DocTestDefaults(), 'runtime')
sage: dt, extras = PSS.create_doctests({})
sage: extras['tab']
['2', '3']

sage: s = "'''\n    sage: import warnings; warnings.warn('foo')\n    doctest:1: UserWarning: foo \n'''"
sage: PSS = PythonStringSource('<runtime>', s, DocTestDefaults(), 'runtime')
sage: dt, extras = PSS.create_doctests({})
sage: extras['line_number']
True
create_doctests(namespace)#

Creates doctests from this string.

INPUT:

OUTPUT:

  • doctests – a list of doctests defined by this string

  • tab_locations – either False or a list of linenumbers on which tabs appear.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import StringDocTestSource, PythonSource
sage: from sage.structure.dynamic_class import dynamic_class
sage: s = "'''\n    sage: 2 + 2\n    4\n'''"
sage: PythonStringSource = dynamic_class('PythonStringSource',(StringDocTestSource, PythonSource))
sage: PSS = PythonStringSource('<runtime>', s, DocTestDefaults(), 'runtime')
sage: dt, tabs = PSS.create_doctests({})
sage: for t in dt:
....:     print("{} {}".format(t.name, t.examples[0].sage_source))
<runtime> 2 + 2
class sage.doctest.sources.TexSource#

Bases: SourceLanguage

This class defines the functions needed for the extraction of doctests from a LaTeX source.

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_paper.tex"
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: type(FDS)
<class 'sage.doctest.sources.TexFileSource'>
ending_docstring(line, check_skip=True)#

Determines whether the input line ends a docstring.

Docstring blocks in tex files are defined by verbatim or lstlisting environments, and can be linked together by adding %link immediately after the end{verbatim} or end{lstlisting}.

Within a verbatim (or lstlisting) block, you can tell Sage not to process the rest of the block by including a %skip line.

INPUT:

  • line – a string, one line of an input file

  • check_skip – boolean (default True), used internally in starting_docstring.

OUTPUT:

  • a boolean giving whether the input line marks the end of a docstring (verbatim block).

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_paper.tex"
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS._init()
sage: FDS.ending_docstring(r"\end{verbatim}")
True
sage: FDS.ending_docstring(r"\end{lstlisting}")
True
sage: FDS.linking
False

Use %link to link with the next verbatim block:

sage: FDS.ending_docstring(r"\end{verbatim}%link")
True
sage: FDS.linking
True

%skip also ends a docstring block:

sage: FDS.ending_docstring("%skip")
True
start_finish_can_overlap = False#
starting_docstring(line)#

Determines whether the input line starts a docstring.

Docstring blocks in tex files are defined by verbatim or lstlisting environments, and can be linked together by adding %link immediately after the end{verbatim} or end{lstlisting}.

Within a verbatim (or lstlisting) block, you can tell Sage not to process the rest of the block by including a %skip line.

INPUT:

  • line – a string, one line of an input file

OUTPUT:

  • a boolean giving whether the input line marks the start of a docstring (verbatim block).

EXAMPLES:

sage: from sage.doctest.control import DocTestDefaults
sage: from sage.doctest.sources import FileDocTestSource
sage: filename = "sage_paper.tex"
sage: FDS = FileDocTestSource(filename,DocTestDefaults())
sage: FDS._init()

We start docstrings with begin{verbatim} or begin{lstlisting}:

sage: FDS.starting_docstring(r"\begin{verbatim}")
True
sage: FDS.starting_docstring(r"\begin{lstlisting}")
True
sage: FDS.skipping
False
sage: FDS.ending_docstring("sage: 2+2")
False
sage: FDS.ending_docstring("4")
False

To start ignoring the rest of the verbatim block, use %skip:

sage: FDS.ending_docstring("%skip")
True
sage: FDS.skipping
True
sage: FDS.starting_docstring("sage: raise RuntimeError")
False

You can even pretend to start another verbatim block while skipping:

sage: FDS.starting_docstring(r"\begin{verbatim}")
False
sage: FDS.skipping
True

To stop skipping end the verbatim block:

sage: FDS.starting_docstring(r"\end{verbatim} %link")
False
sage: FDS.skipping
False

Linking works even when the block was ended while skipping:

sage: FDS.linking
True
sage: FDS.starting_docstring(r"\begin{verbatim}")
True
sage.doctest.sources.get_basename(path)#

This function returns the basename of the given path, e.g. sage.doctest.sources or doc.ru.tutorial.tour_advanced

EXAMPLES:

sage: from sage.doctest.sources import get_basename
sage: from sage.env import SAGE_SRC
sage: import os
sage: get_basename(os.path.join(SAGE_SRC, 'sage', 'doctest', 'sources.py'))
'sage.doctest.sources'
sage: get_basename(os.path.join(SAGE_SRC, 'sage', 'structure', 'element.pxd'))
'sage.structure.element.pxd'