Testing for features of the environment at runtime¶
A computation can require a certain package to be installed in the runtime
environment. Abstractly such a package describes a Feature
which can
be tested for at runtime. It can be of various kinds, most prominently an
Executable
in the PATH
, a PythonModule
, or an additional
package for some installed
system such as a GapPackage
.
AUTHORS:
Julian Rüth (2016-04-07): Initial version
Jeroen Demeyer (2018-02-12): Refactoring and clean up
EXAMPLES:
Some generic features are available for common cases. For example, to
test for the existence of a binary, one can use an Executable
feature:
sage: from sage.features import Executable
sage: Executable(name='sh', executable='sh').is_present()
FeatureTestResult('sh', True)
>>> from sage.all import *
>>> from sage.features import Executable
>>> Executable(name='sh', executable='sh').is_present()
FeatureTestResult('sh', True)
Here we test whether the grape GAP package is available:
sage: from sage.features.gap import GapPackage
sage: GapPackage("grape", spkg='gap_packages').is_present() # optional - gap_package_grape
FeatureTestResult('gap_package_grape', True)
>>> from sage.all import *
>>> from sage.features.gap import GapPackage
>>> GapPackage("grape", spkg='gap_packages').is_present() # optional - gap_package_grape
FeatureTestResult('gap_package_grape', True)
Note that a FeatureTestResult
acts like a bool in most contexts:
sage: if Executable(name='sh', executable='sh').is_present(): "present."
'present.'
>>> from sage.all import *
>>> if Executable(name='sh', executable='sh').is_present(): "present."
'present.'
When one wants to raise an error if the feature is not available, one
can use the require
method:
sage: Executable(name='sh', executable='sh').require()
sage: Executable(name='random', executable='randomOochoz6x', spkg='random', url='http://rand.om').require() # optional - sage_spkg
Traceback (most recent call last):
...
FeatureNotPresentError: random is not available.
Executable 'randomOochoz6x' not found on PATH.
...try to run...sage -i random...
Further installation instructions might be available at http://rand.om.
>>> from sage.all import *
>>> Executable(name='sh', executable='sh').require()
>>> Executable(name='random', executable='randomOochoz6x', spkg='random', url='http://rand.om').require() # optional - sage_spkg
Traceback (most recent call last):
...
FeatureNotPresentError: random is not available.
Executable 'randomOochoz6x' not found on PATH.
...try to run...sage -i random...
Further installation instructions might be available at http://rand.om.
As can be seen above, features try to produce helpful error messages.
- class sage.features.CythonFeature(*args, **kwds)[source]¶
Bases:
Feature
A
Feature
which describes the ability to compile and import a particular piece of Cython code.To test the presence of
name
, the cython compiler is run ontest_code
and the resulting module is imported.EXAMPLES:
sage: from sage.features import CythonFeature sage: fabs_test_code = ''' ....: cdef extern from "<math.h>": ....: double fabs(double x) ....: ....: assert fabs(-1) == 1 ....: ''' sage: fabs = CythonFeature("fabs", test_code=fabs_test_code, # needs sage.misc.cython ....: spkg='gcc', url='https://gnu.org', ....: type='standard') sage: fabs.is_present() # needs sage.misc.cython FeatureTestResult('fabs', True)
>>> from sage.all import * >>> from sage.features import CythonFeature >>> fabs_test_code = ''' ... cdef extern from "<math.h>": ... double fabs(double x) ....: >>> assert fabs(-Integer(1)) == Integer(1) ... ''' >>> fabs = CythonFeature("fabs", test_code=fabs_test_code, # needs sage.misc.cython ... spkg='gcc', url='https://gnu.org', ... type='standard') >>> fabs.is_present() # needs sage.misc.cython FeatureTestResult('fabs', True)
Test various failures:
sage: broken_code = '''this is not a valid Cython program!''' sage: broken = CythonFeature("broken", test_code=broken_code) sage: broken.is_present() FeatureTestResult('broken', False)
>>> from sage.all import * >>> broken_code = '''this is not a valid Cython program!''' >>> broken = CythonFeature("broken", test_code=broken_code) >>> broken.is_present() FeatureTestResult('broken', False)
sage: broken_code = '''cdef extern from "no_such_header_file": pass''' sage: broken = CythonFeature("broken", test_code=broken_code) sage: broken.is_present() FeatureTestResult('broken', False)
>>> from sage.all import * >>> broken_code = '''cdef extern from "no_such_header_file": pass''' >>> broken = CythonFeature("broken", test_code=broken_code) >>> broken.is_present() FeatureTestResult('broken', False)
sage: broken_code = '''import no_such_python_module''' sage: broken = CythonFeature("broken", test_code=broken_code) sage: broken.is_present() FeatureTestResult('broken', False)
>>> from sage.all import * >>> broken_code = '''import no_such_python_module''' >>> broken = CythonFeature("broken", test_code=broken_code) >>> broken.is_present() FeatureTestResult('broken', False)
sage: broken_code = '''raise AssertionError("sorry!")''' sage: broken = CythonFeature("broken", test_code=broken_code) sage: broken.is_present() FeatureTestResult('broken', False)
>>> from sage.all import * >>> broken_code = '''raise AssertionError("sorry!")''' >>> broken = CythonFeature("broken", test_code=broken_code) >>> broken.is_present() FeatureTestResult('broken', False)
- class sage.features.Executable(*args, **kwds)[source]¶
Bases:
FileFeature
A feature describing an executable in the
PATH
.In an installation of Sage with
SAGE_LOCAL
different fromSAGE_VENV
, the executable is searched first inSAGE_VENV/bin
, then inSAGE_LOCAL/bin
, then inPATH
.Note
Overwrite
is_functional()
if you also want to check whether the executable shows proper behaviour.Calls to
is_present()
are cached. You might want to cache theExecutable
object to prevent unnecessary calls to the executable.EXAMPLES:
sage: from sage.features import Executable sage: Executable(name='sh', executable='sh').is_present() FeatureTestResult('sh', True) sage: Executable(name='does-not-exist', executable='does-not-exist-xxxxyxyyxyy').is_present() FeatureTestResult('does-not-exist', False)
>>> from sage.all import * >>> from sage.features import Executable >>> Executable(name='sh', executable='sh').is_present() FeatureTestResult('sh', True) >>> Executable(name='does-not-exist', executable='does-not-exist-xxxxyxyyxyy').is_present() FeatureTestResult('does-not-exist', False)
- absolute_filename()[source]¶
The absolute path of the executable as a string.
EXAMPLES:
sage: from sage.features import Executable sage: Executable(name='sh', executable='sh').absolute_filename() '/...bin/sh'
>>> from sage.all import * >>> from sage.features import Executable >>> Executable(name='sh', executable='sh').absolute_filename() '/...bin/sh'
A
FeatureNotPresentError
is raised if the file cannot be found:sage: Executable(name='does-not-exist', executable='does-not-exist-xxxxyxyyxyy').absolute_filename() Traceback (most recent call last): ... sage.features.FeatureNotPresentError: does-not-exist is not available. Executable 'does-not-exist-xxxxyxyyxyy' not found on PATH.
>>> from sage.all import * >>> Executable(name='does-not-exist', executable='does-not-exist-xxxxyxyyxyy').absolute_filename() Traceback (most recent call last): ... sage.features.FeatureNotPresentError: does-not-exist is not available. Executable 'does-not-exist-xxxxyxyyxyy' not found on PATH.
- is_functional()[source]¶
Return whether an executable in the path is functional.
This method is used internally and can be overridden in subclasses in order to implement a feature test. It should not be called directly. Use
Feature.is_present()
instead.EXAMPLES:
The function returns
True
unless explicitly overwritten:sage: from sage.features import Executable sage: Executable(name='sh', executable='sh').is_functional() FeatureTestResult('sh', True)
>>> from sage.all import * >>> from sage.features import Executable >>> Executable(name='sh', executable='sh').is_functional() FeatureTestResult('sh', True)
- class sage.features.Feature(*args, **kwds)[source]¶
Bases:
TrivialUniqueRepresentation
A feature of the runtime environment.
INPUT:
name
– string; name of the feature. This should be suitable as an optional tag for the Sage doctester, i.e., lowercase alphanumeric with underscores (_
) allowed; features that correspond to Python modules/packages may use periods (.
)spkg
– string; name of the SPKG providing the featuredescription
– string (optional); plain English description of the featureurl
– a URL for the upstream package providing the featuretype
– string; one of'standard'
,'optional'
(default),'experimental'
Overwrite
_is_present()
to add feature checks.EXAMPLES:
sage: from sage.features.gap import GapPackage sage: GapPackage("grape", spkg='gap_packages') # indirect doctest Feature('gap_package_grape')
>>> from sage.all import * >>> from sage.features.gap import GapPackage >>> GapPackage("grape", spkg='gap_packages') # indirect doctest Feature('gap_package_grape')
For efficiency, features are unique:
sage: GapPackage("grape") is GapPackage("grape") True
>>> from sage.all import * >>> GapPackage("grape") is GapPackage("grape") True
- hide()[source]¶
Hide this feature. For example this is used when the doctest option
--hide
is set. Setting an installed feature as hidden pretends that it is not available. To revert this useunhide()
.EXAMPLES:
Benzene is an optional SPKG. The following test fails if it is hidden or not installed. Thus, in the second invocation the optional tag is needed:
sage: from sage.features.graph_generators import Benzene sage: Benzene().hide() sage: len(list(graphs.fusenes(2))) # needs sage.graphs Traceback (most recent call last): ... FeatureNotPresentError: benzene is not available. Feature `benzene` is hidden. Use method `unhide` to make it available again. sage: Benzene().unhide() # optional - benzene, needs sage.graphs sage: len(list(graphs.fusenes(2))) # optional - benzene, needs sage.graphs 1
>>> from sage.all import * >>> from sage.features.graph_generators import Benzene >>> Benzene().hide() >>> len(list(graphs.fusenes(Integer(2)))) # needs sage.graphs Traceback (most recent call last): ... FeatureNotPresentError: benzene is not available. Feature `benzene` is hidden. Use method `unhide` to make it available again. >>> Benzene().unhide() # optional - benzene, needs sage.graphs >>> len(list(graphs.fusenes(Integer(2)))) # optional - benzene, needs sage.graphs 1
Return whether
self
is present but currently hidden.EXAMPLES:
sage: from sage.features.sagemath import sage__plot sage: sage__plot().hide() sage: sage__plot().is_hidden() # needs sage.plot True sage: sage__plot().unhide() sage: sage__plot().is_hidden() False
- is_optional()[source]¶
Return whether this feature corresponds to an optional SPKG.
EXAMPLES:
sage: from sage.features.databases import DatabaseCremona sage: DatabaseCremona().is_optional() True
>>> from sage.all import * >>> from sage.features.databases import DatabaseCremona >>> DatabaseCremona().is_optional() True
- is_present()[source]¶
Return whether the feature is present.
OUTPUT:
A
FeatureTestResult
which can be used as a boolean and contains additional information about the feature test.EXAMPLES:
sage: from sage.features.gap import GapPackage sage: GapPackage("grape", spkg='gap_packages').is_present() # optional - gap_package_grape FeatureTestResult('gap_package_grape', True) sage: GapPackage("NOT_A_PACKAGE", spkg='gap_packages').is_present() FeatureTestResult('gap_package_NOT_A_PACKAGE', False)
>>> from sage.all import * >>> from sage.features.gap import GapPackage >>> GapPackage("grape", spkg='gap_packages').is_present() # optional - gap_package_grape FeatureTestResult('gap_package_grape', True) >>> GapPackage("NOT_A_PACKAGE", spkg='gap_packages').is_present() FeatureTestResult('gap_package_NOT_A_PACKAGE', False)
The result is cached:
sage: from sage.features import Feature sage: class TestFeature(Feature): ....: def _is_present(self): ....: print("checking presence") ....: return True sage: TestFeature("test").is_present() checking presence FeatureTestResult('test', True) sage: TestFeature("test").is_present() FeatureTestResult('test', True) sage: TestFeature("other").is_present() checking presence FeatureTestResult('other', True) sage: TestFeature("other").is_present() FeatureTestResult('other', True)
>>> from sage.all import * >>> from sage.features import Feature >>> class TestFeature(Feature): ... def _is_present(self): ... print("checking presence") ... return True >>> TestFeature("test").is_present() checking presence FeatureTestResult('test', True) >>> TestFeature("test").is_present() FeatureTestResult('test', True) >>> TestFeature("other").is_present() checking presence FeatureTestResult('other', True) >>> TestFeature("other").is_present() FeatureTestResult('other', True)
- is_standard()[source]¶
Return whether this feature corresponds to a standard SPKG.
EXAMPLES:
sage: from sage.features.databases import DatabaseCremona sage: DatabaseCremona().is_standard() False
>>> from sage.all import * >>> from sage.features.databases import DatabaseCremona >>> DatabaseCremona().is_standard() False
- joined_features()[source]¶
Return a list of features that
self
is the join of.OUTPUT:
A (possibly empty) list of instances of
Feature
.EXAMPLES:
sage: from sage.features.graphviz import Graphviz sage: Graphviz().joined_features() [Feature('dot'), Feature('neato'), Feature('twopi')] sage: from sage.features.sagemath import sage__rings__function_field sage: sage__rings__function_field().joined_features() [Feature('sage.rings.function_field.function_field_polymod'), Feature('sage.libs.singular'), Feature('sage.libs.singular.singular'), Feature('sage.interfaces.singular')] sage: from sage.features.interfaces import Mathematica sage: Mathematica().joined_features() []
>>> from sage.all import * >>> from sage.features.graphviz import Graphviz >>> Graphviz().joined_features() [Feature('dot'), Feature('neato'), Feature('twopi')] >>> from sage.features.sagemath import sage__rings__function_field >>> sage__rings__function_field().joined_features() [Feature('sage.rings.function_field.function_field_polymod'), Feature('sage.libs.singular'), Feature('sage.libs.singular.singular'), Feature('sage.interfaces.singular')] >>> from sage.features.interfaces import Mathematica >>> Mathematica().joined_features() []
- require()[source]¶
Raise a
FeatureNotPresentError
if the feature is not present.EXAMPLES:
sage: from sage.features.gap import GapPackage sage: GapPackage("ve1EeThu").require() # needs sage.libs.gap Traceback (most recent call last): ... FeatureNotPresentError: gap_package_ve1EeThu is not available. `LoadPackage("ve1EeThu")` evaluated to `fail` in GAP.
>>> from sage.all import * >>> from sage.features.gap import GapPackage >>> GapPackage("ve1EeThu").require() # needs sage.libs.gap Traceback (most recent call last): ... FeatureNotPresentError: gap_package_ve1EeThu is not available. `LoadPackage("ve1EeThu")` evaluated to `fail` in GAP.
- resolution()[source]¶
Return a suggestion on how to make
is_present()
pass if it did not pass.OUTPUT: string
EXAMPLES:
sage: from sage.features import Executable sage: Executable(name='CSDP', spkg='csdp', executable='theta', url='https://github.com/dimpase/csdp').resolution() # optional - sage_spkg '...To install CSDP...you can try to run...sage -i csdp...Further installation instructions might be available at https://github.com/dimpase/csdp.'
>>> from sage.all import * >>> from sage.features import Executable >>> Executable(name='CSDP', spkg='csdp', executable='theta', url='https://github.com/dimpase/csdp').resolution() # optional - sage_spkg '...To install CSDP...you can try to run...sage -i csdp...Further installation instructions might be available at https://github.com/dimpase/csdp.'
- unhide()[source]¶
Revert what
hide()
did.EXAMPLES:
sage: from sage.features.sagemath import sage__plot sage: sage__plot().hide() sage: sage__plot().is_present() FeatureTestResult(‘sage.plot’, False) sage: sage__plot().unhide() # needs sage.plot sage: sage__plot().is_present() # needs sage.plot FeatureTestResult(‘sage.plot’, True)
- exception sage.features.FeatureNotPresentError(feature, reason=None, resolution=None)[source]¶
Bases:
RuntimeError
A missing feature error.
EXAMPLES:
sage: from sage.features import Feature, FeatureTestResult sage: class Missing(Feature): ....: def _is_present(self): ....: return False sage: Missing(name='missing').require() Traceback (most recent call last): ... FeatureNotPresentError: missing is not available.
>>> from sage.all import * >>> from sage.features import Feature, FeatureTestResult >>> class Missing(Feature): ... def _is_present(self): ... return False >>> Missing(name='missing').require() Traceback (most recent call last): ... FeatureNotPresentError: missing is not available.
- property resolution¶
Initialize self. See help(type(self)) for accurate signature.
- class sage.features.FeatureTestResult(feature, is_present, reason=None, resolution=None)[source]¶
Bases:
object
The result of a
Feature.is_present()
call.Behaves like a boolean with some extra data which may explain why a feature is not present and how this may be resolved.
EXAMPLES:
sage: from sage.features.gap import GapPackage sage: presence = GapPackage("NOT_A_PACKAGE").is_present(); presence # indirect doctest FeatureTestResult('gap_package_NOT_A_PACKAGE', False) sage: bool(presence) False
>>> from sage.all import * >>> from sage.features.gap import GapPackage >>> presence = GapPackage("NOT_A_PACKAGE").is_present(); presence # indirect doctest FeatureTestResult('gap_package_NOT_A_PACKAGE', False) >>> bool(presence) False
Explanatory messages might be available as
reason
andresolution
:sage: presence.reason # needs sage.libs.gap '`LoadPackage("NOT_A_PACKAGE")` evaluated to `fail` in GAP.' sage: bool(presence.resolution) False
>>> from sage.all import * >>> presence.reason # needs sage.libs.gap '`LoadPackage("NOT_A_PACKAGE")` evaluated to `fail` in GAP.' >>> bool(presence.resolution) False
If a feature is not present,
resolution
defaults tofeature.resolution()
if this is defined. If you do not want to use this default you need explicitly setresolution
to a string:sage: from sage.features import FeatureTestResult sage: package = GapPackage("NOT_A_PACKAGE", spkg='no_package') sage: str(FeatureTestResult(package, True).resolution) # optional - sage_spkg '...To install gap_package_NOT_A_PACKAGE...you can try to run...sage -i no_package...' sage: str(FeatureTestResult(package, False).resolution) # optional - sage_spkg '...To install gap_package_NOT_A_PACKAGE...you can try to run...sage -i no_package...' sage: FeatureTestResult(package, False, resolution='rtm').resolution 'rtm'
>>> from sage.all import * >>> from sage.features import FeatureTestResult >>> package = GapPackage("NOT_A_PACKAGE", spkg='no_package') >>> str(FeatureTestResult(package, True).resolution) # optional - sage_spkg '...To install gap_package_NOT_A_PACKAGE...you can try to run...sage -i no_package...' >>> str(FeatureTestResult(package, False).resolution) # optional - sage_spkg '...To install gap_package_NOT_A_PACKAGE...you can try to run...sage -i no_package...' >>> FeatureTestResult(package, False, resolution='rtm').resolution 'rtm'
- property resolution¶
Initialize self. See help(type(self)) for accurate signature.
- class sage.features.FileFeature(*args, **kwds)[source]¶
Bases:
Feature
Base class for features that describe a file or directory in the file system.
A subclass should implement a method
absolute_filename()
.EXAMPLES:
Two direct concrete subclasses of
FileFeature
are defined:sage: from sage.features import StaticFile, Executable, FileFeature sage: issubclass(StaticFile, FileFeature) True sage: issubclass(Executable, FileFeature) True
>>> from sage.all import * >>> from sage.features import StaticFile, Executable, FileFeature >>> issubclass(StaticFile, FileFeature) True >>> issubclass(Executable, FileFeature) True
To work with the file described by the feature, use the method
absolute_filename()
. AFeatureNotPresentError
is raised if the file cannot be found:sage: Executable(name='does-not-exist', executable='does-not-exist-xxxxyxyyxyy').absolute_filename() Traceback (most recent call last): ... sage.features.FeatureNotPresentError: does-not-exist is not available. Executable 'does-not-exist-xxxxyxyyxyy' not found on PATH.
>>> from sage.all import * >>> Executable(name='does-not-exist', executable='does-not-exist-xxxxyxyyxyy').absolute_filename() Traceback (most recent call last): ... sage.features.FeatureNotPresentError: does-not-exist is not available. Executable 'does-not-exist-xxxxyxyyxyy' not found on PATH.
A
FileFeature
also provides theis_present()
method to test for the presence of the file at run time. This is inherited from the base classFeature
:sage: Executable(name='sh', executable='sh').is_present() FeatureTestResult('sh', True)
>>> from sage.all import * >>> Executable(name='sh', executable='sh').is_present() FeatureTestResult('sh', True)
- class sage.features.PythonModule(*args, **kwds)[source]¶
Bases:
Feature
A
Feature
which describes whether a python module can be imported.EXAMPLES:
Not all builds of python include the
ssl
module, so you could check whether it is available:sage: from sage.features import PythonModule sage: PythonModule("ssl").require() # not tested - output depends on the python build
>>> from sage.all import * >>> from sage.features import PythonModule >>> PythonModule("ssl").require() # not tested - output depends on the python build
- class sage.features.StaticFile(*args, **kwds)[source]¶
Bases:
FileFeature
A
Feature
which describes the presence of a certain file such as a database.EXAMPLES:
sage: from sage.features import StaticFile sage: StaticFile(name='no_such_file', filename='KaT1aihu', # optional - sage_spkg ....: search_path='/', spkg='some_spkg', ....: url='http://rand.om').require() Traceback (most recent call last): ... FeatureNotPresentError: no_such_file is not available. 'KaT1aihu' not found in any of ['/']... To install no_such_file...you can try to run...sage -i some_spkg... Further installation instructions might be available at http://rand.om.
>>> from sage.all import * >>> from sage.features import StaticFile >>> StaticFile(name='no_such_file', filename='KaT1aihu', # optional - sage_spkg ... search_path='/', spkg='some_spkg', ... url='http://rand.om').require() Traceback (most recent call last): ... FeatureNotPresentError: no_such_file is not available. 'KaT1aihu' not found in any of ['/']... To install no_such_file...you can try to run...sage -i some_spkg... Further installation instructions might be available at http://rand.om.
- absolute_filename()[source]¶
The absolute path of the file as a string.
EXAMPLES:
sage: from sage.features import StaticFile sage: from sage.misc.temporary_file import tmp_dir sage: dir_with_file = tmp_dir() sage: file_path = os.path.join(dir_with_file, "file.txt") sage: open(file_path, 'a').close() # make sure the file exists sage: search_path = ( '/foo/bar', dir_with_file ) # file is somewhere in the search path sage: feature = StaticFile(name='file', filename='file.txt', search_path=search_path) sage: feature.absolute_filename() == file_path True
>>> from sage.all import * >>> from sage.features import StaticFile >>> from sage.misc.temporary_file import tmp_dir >>> dir_with_file = tmp_dir() >>> file_path = os.path.join(dir_with_file, "file.txt") >>> open(file_path, 'a').close() # make sure the file exists >>> search_path = ( '/foo/bar', dir_with_file ) # file is somewhere in the search path >>> feature = StaticFile(name='file', filename='file.txt', search_path=search_path) >>> feature.absolute_filename() == file_path True
A
FeatureNotPresentError
is raised if the file cannot be found:sage: from sage.features import StaticFile sage: StaticFile(name='no_such_file', filename='KaT1aihu',\ search_path=(), spkg='some_spkg',\ url='http://rand.om').absolute_filename() # optional - sage_spkg Traceback (most recent call last): ... FeatureNotPresentError: no_such_file is not available. 'KaT1aihu' not found in any of []... To install no_such_file...you can try to run...sage -i some_spkg... Further installation instructions might be available at http://rand.om.
>>> from sage.all import * >>> from sage.features import StaticFile >>> StaticFile(name='no_such_file', filename='KaT1aihu',\ search_path=(), spkg='some_spkg',\ url='http://rand.om').absolute_filename() # optional - sage_spkg Traceback (most recent call last): ... FeatureNotPresentError: no_such_file is not available. 'KaT1aihu' not found in any of []... To install no_such_file...you can try to run...sage -i some_spkg... Further installation instructions might be available at http://rand.om.
- class sage.features.TrivialClasscallMetaClass[source]¶
Bases:
type
A trivial version of
sage.misc.classcall_metaclass.ClasscallMetaclass
without Cython dependencies.
- class sage.features.TrivialUniqueRepresentation(*args, **kwds)[source]¶
Bases:
object
A trivial version of
UniqueRepresentation
without Cython dependencies.
- sage.features.package_systems()[source]¶
Return a list of
PackageSystem
objects representing the available package systems.The list is ordered by decreasing preference.
EXAMPLES:
sage: from sage.features import package_systems sage: package_systems() # random [Feature('homebrew'), Feature('sage_spkg'), Feature('pip')]
>>> from sage.all import * >>> from sage.features import package_systems >>> package_systems() # random [Feature('homebrew'), Feature('sage_spkg'), Feature('pip')]