The Sage Preparser¶

EXAMPLES:

Preparsing:

Sage

sage: preparse('2/3')
'Integer(2)/Integer(3)'
sage: preparse('2.5')
"RealNumber('2.5')"
sage: preparse('2^3')
'Integer(2)**Integer(3)'
sage: preparse('a^b')            # exponent
'a**b'
sage: preparse('a**b')
'a**b'
sage: preparse('G.0')            # generator
'G.gen(0)'
sage: preparse('a = 939393R')    # raw
'a = 939393'
sage: implicit_multiplication(True)
sage: preparse('a b c in L')     # implicit multiplication
'a*b*c in L'
sage: preparse('2e3x + 3exp(y)')
"RealNumber('2e3')*x + Integer(3)*exp(y)"

Python

>>> from sage.all import *
>>> preparse('2/3')
'Integer(2)/Integer(3)'
>>> preparse('2.5')
"RealNumber('2.5')"
>>> preparse('2^3')
'Integer(2)**Integer(3)'
>>> preparse('a^b')            # exponent
'a**b'
>>> preparse('a**b')
'a**b'
>>> preparse('G.0')            # generator
'G.gen(0)'
>>> preparse('a = 939393R')    # raw
'a = 939393'
>>> implicit_multiplication(True)
>>> preparse('a b c in L')     # implicit multiplication
'a*b*c in L'
>>> preparse('2e3x + 3exp(y)')
"RealNumber('2e3')*x + Integer(3)*exp(y)"

A string with escaped quotes in it (the point here is that the preparser does not get confused by the internal quotes):

Sage

sage: ""Yes," he said."
'"Yes," he said.'
sage: s = "\"; s
'\'

Python

>>> from sage.all import *
>>> ""Yes," he said."
'"Yes," he said.'
>>> s = "\"; s
'\'

A hex literal:

Sage

sage: preparse('0x2e3')
'Integer(0x2e3)'
sage: 0xA
10
sage: 0xe
14

Python

>>> from sage.all import *
>>> preparse('0x2e3')
'Integer(0x2e3)'
>>> Integer(0xA)
10
>>> Integer(0xe)
14

Raw and hex work correctly:

Sage

sage: type(0xa1)
<class 'sage.rings.integer.Integer'>
sage: type(0xa1r)
<class 'int'>
sage: type(0Xa1R)
<class 'int'>

Python

>>> from sage.all import *
>>> type(Integer(0xa1))
<class 'sage.rings.integer.Integer'>
>>> type(0xa1)
<class 'int'>
>>> type(0Xa1)
<class 'int'>

The preparser can handle PEP 515 (see Issue #28490):

Sage

sage: 1_000_000 + 3_000
1003000

Python

>>> from sage.all import *
>>> Integer(1_000_000) + Integer(3_000)
1003000

In Sage, methods can also be called on integer and real literals (note that in pure Python this would be a syntax error):

Sage

sage: 16.sqrt()
4
sage: 87.factor()
3 * 29
sage: 15.10.sqrt()                                                                  # needs sage.rings.real_mpfr
3.88587184554509
sage: preparse('87.sqrt()')
'Integer(87).sqrt()'
sage: preparse('15.10.sqrt()')
"RealNumber('15.10').sqrt()"

Python

>>> from sage.all import *
>>> Integer(16).sqrt()
4
>>> Integer(87).factor()
3 * 29
>>> RealNumber('15.10').sqrt()                                                                  # needs sage.rings.real_mpfr
3.88587184554509
>>> preparse('87.sqrt()')
'Integer(87).sqrt()'
>>> preparse('15.10.sqrt()')
"RealNumber('15.10').sqrt()"

Note that calling methods on int literals in pure Python is a syntax error, but Sage allows this for Sage integers and reals, because users frequently request it:

Sage

sage: eval('4.__add__(3)')
Traceback (most recent call last):
...
SyntaxError: invalid ...

Python

>>> from sage.all import *
>>> eval('4.__add__(3)')
Traceback (most recent call last):
...
SyntaxError: invalid ...

Symbolic functional notation:

Sage

sage: # needs sage.symbolic
sage: a = 10; f(theta, beta) = theta + beta; b = x^2 + theta
sage: f
(theta, beta) |--> beta + theta
sage: a
10
sage: b
x^2 + theta
sage: f(theta,theta)
2*theta

sage: a = 5; f(x,y) = x*y*sqrt(a)                                                   # needs sage.symbolic
sage: f                                                                             # needs sage.symbolic
(x, y) |--> sqrt(5)*x*y

Python

>>> from sage.all import *
>>> # needs sage.symbolic
>>> a = Integer(10); __tmp__=var("theta,beta"); f = symbolic_expression(theta + beta).function(theta,beta); b = x**Integer(2) + theta
>>> f
(theta, beta) |--> beta + theta
>>> a
10
>>> b
x^2 + theta
>>> f(theta,theta)
2*theta

>>> a = Integer(5); __tmp__=var("x,y"); f = symbolic_expression(x*y*sqrt(a)                                                   ).function(x,y)# needs sage.symbolic
>>> f                                                                             # needs sage.symbolic
(x, y) |--> sqrt(5)*x*y

This involves an =-, but should still be turned into a symbolic expression:

Sage

sage: preparse('a(x) =- 5')
'__tmp__=var("x"); a = symbolic_expression(- Integer(5)).function(x)'
sage: f(x)=-x                                                                       # needs sage.symbolic
sage: f(10)                                                                         # needs sage.symbolic
-10

Python

>>> from sage.all import *
>>> preparse('a(x) =- 5')
'__tmp__=var("x"); a = symbolic_expression(- Integer(5)).function(x)'
>>> __tmp__=var("x"); f = symbolic_expression(-x                                                                       ).function(x)# needs sage.symbolic
>>> f(Integer(10))                                                                         # needs sage.symbolic
-10

This involves -=, which should not be turned into a symbolic expression (of course a(x) is not an identifier, so this will never be valid):

Sage

sage: preparse('a(x) -= 5')
'a(x) -= Integer(5)'

Python

>>> from sage.all import *
>>> preparse('a(x) -= 5')
'a(x) -= Integer(5)'

Raw literals:

Raw literals are not preparsed, which can be useful from an efficiency point of view. In Sage raw integer and floating literals are followed by an”r” (or “R”) for raw, meaning not preparsed.

We create a raw integer:

Sage

sage: a = 393939r
sage: a
393939
sage: type(a)
<class 'int'>

Python

>>> from sage.all import *
>>> a = 393939
>>> a
393939
>>> type(a)
<class 'int'>

We create a raw float:

Sage

sage: z = 1.5949r
sage: z
1.5949
sage: type(z)
<class 'float'>

Python

>>> from sage.all import *
>>> z = 1.5949
>>> z
1.5949
>>> type(z)
<class 'float'>

You can also use an upper case letter:

Sage

sage: z = 3.1415R
sage: z
3.1415
sage: type(z)
<class 'float'>

Python

>>> from sage.all import *
>>> z = 3.1415
>>> z
3.1415
>>> type(z)
<class 'float'>

This next example illustrates how raw literals can be very useful in certain cases. We make a list of even integers up to 10000:

Sage

sage: v = [ 2*i for i in range(10000)]

Python

>>> from sage.all import *
>>> v = [ Integer(2)*i for i in range(Integer(10000))]

This takes a noticeable fraction of a second (e.g., 0.25 seconds). After preparsing, what Python is really executing is the following:

Sage

sage: preparse('v = [ 2*i for i in range(10000)]')
'v = [ Integer(2)*i for i in range(Integer(10000))]'

Python

>>> from sage.all import *
>>> preparse('v = [ 2*i for i in range(10000)]')
'v = [ Integer(2)*i for i in range(Integer(10000))]'

If instead we use a raw 2 we get execution that is instant (0.00 seconds):

Sage

sage: v = [ 2r * i for i in range(10000r)]

Python

>>> from sage.all import *
>>> v = [ 2 * i for i in range(10000)]

Behind the scenes what happens is the following:

Sage

sage: preparse('v = [ 2r * i for i in range(10000r)]')
'v = [ 2 * i for i in range(10000)]'

Python

>>> from sage.all import *
>>> preparse('v = [ 2r * i for i in range(10000r)]')
'v = [ 2 * i for i in range(10000)]'

Warning

The results of the above two expressions are different. The first one computes a list of Sage integers, whereas the second creates a list of Python integers. Python integers are typically much more efficient than Sage integers when they are very small; large Sage integers are much more efficient than Python integers, since they are implemented using the GMP C library.

F-Strings (PEP 498):

Expressions embedded within F-strings are preparsed:

Sage

sage: f'{1/3}'
'1/3'
sage: f'{2^3}'
'8'
sage: x = 20
sage: f'{x} in binary is: {x:08b}'
'20 in binary is: 00010100'
sage: f'{list(map(lambda x: x^2, [1, 2, .., 5]))}'
'[1, 4, 9, 16, 25]'

Python

>>> from sage.all import *
>>> f'{Integer(1)/Integer(3)}'
'1/3'
>>> f'{Integer(2)**Integer(3)}'
'8'
>>> x = Integer(20)
>>> f'{x} in binary is: {x:08b}'
'20 in binary is: 00010100'
>>> f'{list(map(lambda x: x**Integer(2), (ellipsis_range(Integer(1), Integer(2),Ellipsis, Integer(5)))))}'
'[1, 4, 9, 16, 25]'

Note that the format specifier is not preparsed. Expressions within it, however, are:

Sage

sage: f'{x:10r}'
Traceback (most recent call last):
...
ValueError: Unknown format code 'r' for object of type 'int'
sage: f'{x:{10r}}'
'        20'

Python

>>> from sage.all import *
>>> f'{x:10r}'
Traceback (most recent call last):
...
ValueError: Unknown format code 'r' for object of type 'int'
>>> f'{x:{10}}'
'        20'

Nested F-strings are also supported:

Sage

sage: f'{ f"{ 1/3 + 1/6 }" }'
'1/2'
sage: f'''1{ f"2{ f'4{ 2^3 }4' }2" }1'''
'1248421'

Python

>>> from sage.all import *
>>> f'{ f"{ Integer(1)/Integer(3) + Integer(1)/Integer(6) }" }'
'1/2'
>>> f'''1{ f"2{ f'4{ Integer(2)**Integer(3) }4' }2" }1'''
'1248421'

AUTHORS:

William Stein (2006-02-19): fixed bug when loading .py files
William Stein (2006-03-09): fixed crash in parsing exponentials; precision of real literals now determined by digits of input (like Mathematica)
Joe Wetherell (2006-04-14): added MAGMA-style constructor preparsing
Bobby Moretti (2007-01-25): added preliminary function assignment notation
Robert Bradshaw (2007-09-19): added strip_string_literals, containing_block utility functions. Arrr!; added [1,2,..,n] notation
Robert Bradshaw (2008-01-04): implicit multiplication (off by default)
Robert Bradshaw (2008-09-23): factor out constants
Robert Bradshaw (2009-01): simplify preparser by making it modular and using regular expressions; bug fixes, complex numbers, and binary input

class sage.repl.preparse.QuoteStack[source]¶

Bases: object

The preserved state of parsing in strip_string_literals().

peek()[source]¶

Get the frame at the top of the stack or None if empty.

EXAMPLES:

Sage

sage: qs = sage.repl.preparse.QuoteStack()
sage: qs.peek()
sage: qs.push(sage.repl.preparse.QuoteStackFrame('"'))
sage: qs.peek()
QuoteStackFrame(...delim='"'...)

Python

>>> from sage.all import *
>>> qs = sage.repl.preparse.QuoteStack()
>>> qs.peek()
>>> qs.push(sage.repl.preparse.QuoteStackFrame('"'))
>>> qs.peek()
QuoteStackFrame(...delim='"'...)

pop()[source]¶

Remove and return the frame that was most recently added to the stack.

Raise an IndexError if the stack is empty.

EXAMPLES:

Sage

sage: qs = sage.repl.preparse.QuoteStack()
sage: qs.pop()
Traceback (most recent call last):
...
IndexError: ...
sage: qs.push(sage.repl.preparse.QuoteStackFrame('"'))
sage: qs.pop()
QuoteStackFrame(...delim='"'...)

Python

>>> from sage.all import *
>>> qs = sage.repl.preparse.QuoteStack()
>>> qs.pop()
Traceback (most recent call last):
...
IndexError: ...
>>> qs.push(sage.repl.preparse.QuoteStackFrame('"'))
>>> qs.pop()
QuoteStackFrame(...delim='"'...)

push(frame)[source]¶

Add a frame to the stack.

If the frame corresponds to an F-string, its delimiter is marked as no longer being a safe_delimiter().

EXAMPLES:

Sage

sage: qs = sage.repl.preparse.QuoteStack()
sage: qs.push(sage.repl.preparse.QuoteStackFrame("'"))
sage: len(qs)
1

Python

>>> from sage.all import *
>>> qs = sage.repl.preparse.QuoteStack()
>>> qs.push(sage.repl.preparse.QuoteStackFrame("'"))
>>> len(qs)
1

safe_delimiter()[source]¶

Return a string delimiter that may be safely inserted into the code output by strip_string_literals(), if any.

' is preferred over ". The triple-quoted versions are never returned since by the time they would be chosen, they would also be invalid. ''' cannot, for example, appear within an F-string delimited by '.

Once marked unsafe, a delimiter is never made safe again, even after the stack frame that used it is popped. It may no longer be applicable to parsing, but it appears somewhere in the processed code, so it is not safe to insert just anywhere. A future enhancement could be to map ranges in the processed code to the delimiter(s) that would be safe to insert there.

EXAMPLES:

Sage

sage: from sage.repl.preparse import QuoteStack, QuoteStackFrame
sage: s = QuoteStack()
sage: s.safe_delimiter()
"'"
sage: s.push(QuoteStackFrame("'"))
sage: s.safe_delimiter()
"'"
sage: s.pop()
QuoteStackFrame(...)
sage: s.push(QuoteStackFrame("'", f_string=True))
sage: s.safe_delimiter()
'"'
sage: s.push(QuoteStackFrame('"', f_string=True))
sage: s.safe_delimiter() is None
True

Python

>>> from sage.all import *
>>> from sage.repl.preparse import QuoteStack, QuoteStackFrame
>>> s = QuoteStack()
>>> s.safe_delimiter()
"'"
>>> s.push(QuoteStackFrame("'"))
>>> s.safe_delimiter()
"'"
>>> s.pop()
QuoteStackFrame(...)
>>> s.push(QuoteStackFrame("'", f_string=True))
>>> s.safe_delimiter()
'"'
>>> s.push(QuoteStackFrame('"', f_string=True))
>>> s.safe_delimiter() is None
True

class sage.repl.preparse.QuoteStackFrame(delim, raw=False, f_string=False, braces=0, parens=0, brackets=0, fmt_spec=False, nested_fmt_spec=False)[source]¶

Bases: SimpleNamespace

The state of a single level of a string literal being parsed.

Only F-strings have more than one level.

sage.repl.preparse.containing_block(code, idx, delimiters=['()', '[]', '{}'], require_delim=True)[source]¶

Find the code block given by balanced delimiters that contains the position idx.

INPUT:

code – string
idx – integer; a starting position
delimiters – list of strings (default: [‘()’, ‘[]’, ‘{}’]); the delimiters to balance. A delimiter must be a single character and no character can at the same time be opening and closing delimiter.
require_delim – boolean (default: True); whether to raise a SyntaxError if delimiters are present. If the delimiters are unbalanced, an error will be raised in any case.

OUTPUT:

a 2-tuple (a,b) of integers, such that code[a:b] is delimited by balanced delimiters, a<=idx<b, and a is maximal and b is minimal with that property. If that does not exist, a SyntaxError is raised.
If require_delim is false and a,b as above can not be found, then 0, len(code) is returned.

EXAMPLES:

Sage

sage: from sage.repl.preparse import containing_block
sage: s = "factor(next_prime(L[5]+1))"
sage: s[22]
'+'
sage: start, end = containing_block(s, 22)
sage: start, end
(17, 25)
sage: s[start:end]
'(L[5]+1)'
sage: s[20]
'5'
sage: start, end = containing_block(s, 20); s[start:end]
'[5]'
sage: start, end = containing_block(s, 20, delimiters=['()']); s[start:end]
'(L[5]+1)'
sage: start, end = containing_block(s, 10); s[start:end]
'(next_prime(L[5]+1))'

Python

>>> from sage.all import *
>>> from sage.repl.preparse import containing_block
>>> s = "factor(next_prime(L[5]+1))"
>>> s[Integer(22)]
'+'
>>> start, end = containing_block(s, Integer(22))
>>> start, end
(17, 25)
>>> s[start:end]
'(L[5]+1)'
>>> s[Integer(20)]
'5'
>>> start, end = containing_block(s, Integer(20)); s[start:end]
'[5]'
>>> start, end = containing_block(s, Integer(20), delimiters=['()']); s[start:end]
'(L[5]+1)'
>>> start, end = containing_block(s, Integer(10)); s[start:end]
'(next_prime(L[5]+1))'

sage.repl.preparse.extract_numeric_literals(code)[source]¶

Pulls out numeric literals and assigns them to global variables. This eliminates the need to re-parse and create the literals, e.g., during every iteration of a loop.

INPUT:

code – string; a block of code

OUTPUT:

a (string, string:string dictionary) 2-tuple; the block with literals replaced by variable names and a mapping from names to the new variables

EXAMPLES:

Sage

sage: from sage.repl.preparse import extract_numeric_literals
sage: code, nums = extract_numeric_literals("1.2 + 5")
sage: print(code)
_sage_const_1p2  + _sage_const_5
sage: print(nums)
{'_sage_const_1p2': "RealNumber('1.2')", '_sage_const_5': 'Integer(5)'}

sage: extract_numeric_literals("[1, 1.1, 1e1, -1e-1, 1.]")[0]
'[_sage_const_1 , _sage_const_1p1 , _sage_const_1e1 , -_sage_const_1en1 , _sage_const_1p ]'

sage: extract_numeric_literals("[1.sqrt(), 1.2.sqrt(), 1r, 1.2r, R.1, R0.1, (1..5)]")[0]
'[_sage_const_1 .sqrt(), _sage_const_1p2 .sqrt(), 1 , 1.2 , R.1, R0.1, (_sage_const_1 .._sage_const_5 )]'

Python

>>> from sage.all import *
>>> from sage.repl.preparse import extract_numeric_literals
>>> code, nums = extract_numeric_literals("1.2 + 5")
>>> print(code)
_sage_const_1p2  + _sage_const_5
>>> print(nums)
{'_sage_const_1p2': "RealNumber('1.2')", '_sage_const_5': 'Integer(5)'}

>>> extract_numeric_literals("[1, 1.1, 1e1, -1e-1, 1.]")[Integer(0)]
'[_sage_const_1 , _sage_const_1p1 , _sage_const_1e1 , -_sage_const_1en1 , _sage_const_1p ]'

>>> extract_numeric_literals("[1.sqrt(), 1.2.sqrt(), 1r, 1.2r, R.1, R0.1, (1..5)]")[Integer(0)]
'[_sage_const_1 .sqrt(), _sage_const_1p2 .sqrt(), 1 , 1.2 , R.1, R0.1, (_sage_const_1 .._sage_const_5 )]'

sage.repl.preparse.handle_encoding_declaration(contents, out)[source]¶

Find a PEP 263-style Python encoding declaration in the first or second line of contents. If found, output it to out and return contents without the encoding line; otherwise output a default UTF-8 declaration and return contents.

EXAMPLES:

Sage

sage: from sage.repl.preparse import handle_encoding_declaration
sage: import sys
sage: c1='# -*- coding: latin-1 -*-\nimport os, sys\n...'
sage: c2='# -*- coding: iso-8859-15 -*-\nimport os, sys\n...'
sage: c3='# -*- coding: ascii -*-\nimport os, sys\n...'
sage: c4='import os, sys\n...'
sage: handle_encoding_declaration(c1, sys.stdout)
# -*- coding: latin-1 -*-
'import os, sys\n...'
sage: handle_encoding_declaration(c2, sys.stdout)
# -*- coding: iso-8859-15 -*-
'import os, sys\n...'
sage: handle_encoding_declaration(c3, sys.stdout)
# -*- coding: ascii -*-
'import os, sys\n...'
sage: handle_encoding_declaration(c4, sys.stdout)
# -*- coding: utf-8 -*-
'import os, sys\n...'

Python

>>> from sage.all import *
>>> from sage.repl.preparse import handle_encoding_declaration
>>> import sys
>>> c1='# -*- coding: latin-1 -*-\nimport os, sys\n...'
>>> c2='# -*- coding: iso-8859-15 -*-\nimport os, sys\n...'
>>> c3='# -*- coding: ascii -*-\nimport os, sys\n...'
>>> c4='import os, sys\n...'
>>> handle_encoding_declaration(c1, sys.stdout)
# -*- coding: latin-1 -*-
'import os, sys\n...'
>>> handle_encoding_declaration(c2, sys.stdout)
# -*- coding: iso-8859-15 -*-
'import os, sys\n...'
>>> handle_encoding_declaration(c3, sys.stdout)
# -*- coding: ascii -*-
'import os, sys\n...'
>>> handle_encoding_declaration(c4, sys.stdout)
# -*- coding: utf-8 -*-
'import os, sys\n...'

Note

PEP 263 says that Python will interpret a UTF-8 byte order mark as a declaration of UTF-8 encoding, but I do not think we do that; this function only sees a Python string so it cannot account for a BOM.
We default to UTF-8 encoding even though PEP 263 says that Python files should default to ASCII.
Also see https://docs.python.org/ref/encodings.html.

AUTHORS:

Lars Fischer
Dan Drake (2010-12-08, rewrite for Issue #10440)

sage.repl.preparse.implicit_mul(code, level=5)[source]¶

Insert *’s to make implicit multiplication explicit.

INPUT:

code – string; the code with missing *’s
level – integer (default: 5); see implicit_multiplication() for a list

OUTPUT: string

EXAMPLES:

Sage

sage: from sage.repl.preparse import implicit_mul
sage: implicit_mul('(2x^2-4x+3)a0')
'(2*x^2-4*x+3)*a0'
sage: implicit_mul('a b c in L')
'a*b*c in L'
sage: implicit_mul('1r + 1e3 + 5exp(2)')
'1r + 1e3 + 5*exp(2)'
sage: implicit_mul('f(a)(b)', level=10)
'f(a)*(b)'

Python

>>> from sage.all import *
>>> from sage.repl.preparse import implicit_mul
>>> implicit_mul('(2x^2-4x+3)a0')
'(2*x^2-4*x+3)*a0'
>>> implicit_mul('a b c in L')
'a*b*c in L'
>>> implicit_mul('1r + 1e3 + 5exp(2)')
'1r + 1e3 + 5*exp(2)'
>>> implicit_mul('f(a)(b)', level=Integer(10))
'f(a)*(b)'

sage.repl.preparse.implicit_multiplication(level=None)[source]¶

Turn implicit multiplication on or off, optionally setting a specific level.

INPUT:

level – boolean or integer (default: 5); how aggressive to be in placing *’s
- 0 – Do nothing
- 1 – Numeric followed by alphanumeric
- 2 – Closing parentheses followed by alphanumeric
- 3 – Spaces between alphanumeric
- 10 – Adjacent parentheses (may mangle call statements)

OUTPUT: the current level if no argument is given

EXAMPLES:

Sage

sage: implicit_multiplication(True)
sage: implicit_multiplication()
5
sage: preparse('2x')
'Integer(2)*x'
sage: implicit_multiplication(False)
sage: preparse('2x')
'2x'

Python

>>> from sage.all import *
>>> implicit_multiplication(True)
>>> implicit_multiplication()
5
>>> preparse('2x')
'Integer(2)*x'
>>> implicit_multiplication(False)
>>> preparse('2x')
'2x'

Note that the IPython automagic feature cannot be used if level >= 3:

Sage

sage: implicit_multiplication(3)
sage: preparse('cd Documents')
'cd*Documents'
sage: implicit_multiplication(2)
sage: preparse('cd Documents')
'cd Documents'
sage: implicit_multiplication(False)

Python

>>> from sage.all import *
>>> implicit_multiplication(Integer(3))
>>> preparse('cd Documents')
'cd*Documents'
>>> implicit_multiplication(Integer(2))
>>> preparse('cd Documents')
'cd Documents'
>>> implicit_multiplication(False)

In this case, one can use the explicit syntax for IPython magics such as %cd Documents.

sage.repl.preparse.in_quote()[source]¶

sage.repl.preparse.isalphadigit_(s)[source]¶

Return True if s is a non-empty string of alphabetic characters or a non-empty string of digits or just a single _

EXAMPLES:

Sage

sage: from sage.repl.preparse import isalphadigit_
sage: isalphadigit_('abc')
True
sage: isalphadigit_('123')
True
sage: isalphadigit_('_')
True
sage: isalphadigit_('a123')
False

Python

>>> from sage.all import *
>>> from sage.repl.preparse import isalphadigit_
>>> isalphadigit_('abc')
True
>>> isalphadigit_('123')
True
>>> isalphadigit_('_')
True
>>> isalphadigit_('a123')
False

sage.repl.preparse.parse_ellipsis(code, preparse_step=True)[source]¶

Preparses [0,2,..,n] notation.

INPUT:

code – string
preparse_step – boolean (default: True)

OUTPUT: string

EXAMPLES:

Sage

sage: from sage.repl.preparse import parse_ellipsis
sage: parse_ellipsis("[1,2,..,n]")
'(ellipsis_range(1,2,Ellipsis,n))'
sage: parse_ellipsis("for i in (f(x) .. L[10]):")
'for i in (ellipsis_iter(f(x) ,Ellipsis, L[10])):'
sage: [1.0..2.0]                                                                # needs sage.rings.real_mpfr
[1.00000000000000, 2.00000000000000]

Python

>>> from sage.all import *
>>> from sage.repl.preparse import parse_ellipsis
>>> parse_ellipsis("[1,2,..,n]")
'(ellipsis_range(1,2,Ellipsis,n))'
>>> parse_ellipsis("for i in (f(x) .. L[10]):")
'for i in (ellipsis_iter(f(x) ,Ellipsis, L[10])):'
>>> (ellipsis_range(RealNumber('1.0'),Ellipsis,RealNumber('2.0')))                                                                # needs sage.rings.real_mpfr
[1.00000000000000, 2.00000000000000]

sage.repl.preparse.preparse(line, reset=True, do_time=False, ignore_prompts=False, numeric_literals=True)[source]¶

Preparse a line of input.

INPUT:

line – string
reset – boolean (default: True)
do_time – boolean (default: False)
ignore_prompts – boolean (default: False)
numeric_literals – boolean (default: True)

OUTPUT: string

EXAMPLES:

Sage

sage: preparse("ZZ.<x> = ZZ['x']")
"ZZ = ZZ['x']; (x,) = ZZ._first_ngens(1)"
sage: preparse("ZZ.<x> = ZZ['y']")
"ZZ = ZZ['y']; (x,) = ZZ._first_ngens(1)"
sage: preparse("ZZ.<x,y> = ZZ[]")
"ZZ = ZZ['x, y']; (x, y,) = ZZ._first_ngens(2)"
sage: preparse("ZZ.<x,y> = ZZ['u,v']")
"ZZ = ZZ['u,v']; (x, y,) = ZZ._first_ngens(2)"
sage: preparse("ZZ.<x> = QQ[2^(1/3)]")
'ZZ = QQ[Integer(2)**(Integer(1)/Integer(3))]; (x,) = ZZ._first_ngens(1)'
sage: QQ[2^(1/3)]                                                               # needs sage.rings.number_field sage.symbolic
Number Field in a with defining polynomial x^3 - 2 with a = 1.259921049894873?

sage: preparse("a^b")
'a**b'
sage: preparse("a^^b")
'a^b'
sage: 8^1
8
sage: 8^^1
9
sage: 9^^1
8

sage: preparse("A \\ B")
'A  * BackslashOperator() * B'
sage: preparse("A^2 \\ B + C")
'A**Integer(2)  * BackslashOperator() * B + C'
sage: preparse("a \\ b \\") # There is really only one backslash here, it is just being escaped.
'a  * BackslashOperator() * b \\'

sage: preparse("time R.<x> = ZZ[]", do_time=True)
'__time__ = cputime(); __wall__ = walltime(); R = ZZ[\'x\']; print("Time: CPU {:.2f} s, Wall: {:.2f} s".format(cputime(__time__), walltime(__wall__))); (x,) = R._first_ngens(1)'

Python

>>> from sage.all import *
>>> preparse("ZZ.<x> = ZZ['x']")
"ZZ = ZZ['x']; (x,) = ZZ._first_ngens(1)"
>>> preparse("ZZ.<x> = ZZ['y']")
"ZZ = ZZ['y']; (x,) = ZZ._first_ngens(1)"
>>> preparse("ZZ.<x,y> = ZZ[]")
"ZZ = ZZ['x, y']; (x, y,) = ZZ._first_ngens(2)"
>>> preparse("ZZ.<x,y> = ZZ['u,v']")
"ZZ = ZZ['u,v']; (x, y,) = ZZ._first_ngens(2)"
>>> preparse("ZZ.<x> = QQ[2^(1/3)]")
'ZZ = QQ[Integer(2)**(Integer(1)/Integer(3))]; (x,) = ZZ._first_ngens(1)'
>>> QQ[Integer(2)**(Integer(1)/Integer(3))]                                                               # needs sage.rings.number_field sage.symbolic
Number Field in a with defining polynomial x^3 - 2 with a = 1.259921049894873?

>>> preparse("a^b")
'a**b'
>>> preparse("a^^b")
'a^b'
>>> Integer(8)**Integer(1)
8
>>> Integer(8)^Integer(1)
9
>>> Integer(9)^Integer(1)
8

>>> preparse("A \\ B")
'A  * BackslashOperator() * B'
>>> preparse("A^2 \\ B + C")
'A**Integer(2)  * BackslashOperator() * B + C'
>>> preparse("a \\ b \\") # There is really only one backslash here, it is just being escaped.
'a  * BackslashOperator() * b \\'

>>> preparse("time R.<x> = ZZ[]", do_time=True)
'__time__ = cputime(); __wall__ = walltime(); R = ZZ[\'x\']; print("Time: CPU {:.2f} s, Wall: {:.2f} s".format(cputime(__time__), walltime(__wall__))); (x,) = R._first_ngens(1)'

sage.repl.preparse.preparse_calculus(code)[source]¶

Supports calculus-like function assignment, e.g., transforms:

f(x,y,z) = sin(x^3 - 4*y) + y^x

into:

__tmp__=var("x,y,z")
f = symbolic_expression(sin(x**3 - 4*y) + y**x).function(x,y,z)

AUTHORS:

Bobby Moretti
- Initial version - 02/2007
William Stein
- Make variables become defined if they are not already defined.
Robert Bradshaw
- Rewrite using regular expressions (01/2009)

EXAMPLES:

Sage

sage: preparse("f(x) = x^3-x")
'__tmp__=var("x"); f = symbolic_expression(x**Integer(3)-x).function(x)'
sage: preparse("f(u,v) = u - v")
'__tmp__=var("u,v"); f = symbolic_expression(u - v).function(u,v)'
sage: preparse("f(x) =-5")
'__tmp__=var("x"); f = symbolic_expression(-Integer(5)).function(x)'
sage: preparse("f(x) -= 5")
'f(x) -= Integer(5)'
sage: preparse("f(x_1, x_2) = x_1^2 - x_2^2")
'__tmp__=var("x_1,x_2"); f = symbolic_expression(x_1**Integer(2) - x_2**Integer(2)).function(x_1,x_2)'

Python

>>> from sage.all import *
>>> preparse("f(x) = x^3-x")
'__tmp__=var("x"); f = symbolic_expression(x**Integer(3)-x).function(x)'
>>> preparse("f(u,v) = u - v")
'__tmp__=var("u,v"); f = symbolic_expression(u - v).function(u,v)'
>>> preparse("f(x) =-5")
'__tmp__=var("x"); f = symbolic_expression(-Integer(5)).function(x)'
>>> preparse("f(x) -= 5")
'f(x) -= Integer(5)'
>>> preparse("f(x_1, x_2) = x_1^2 - x_2^2")
'__tmp__=var("x_1,x_2"); f = symbolic_expression(x_1**Integer(2) - x_2**Integer(2)).function(x_1,x_2)'

For simplicity, this function assumes all statements begin and end with a semicolon:

Sage

sage: from sage.repl.preparse import preparse_calculus
sage: preparse_calculus(";f(t,s)=t^2;")
';__tmp__=var("t,s"); f = symbolic_expression(t^2).function(t,s);'
sage: preparse_calculus(";f( t , s ) = t^2;")
';__tmp__=var("t,s"); f = symbolic_expression(t^2).function(t,s);'

Python

>>> from sage.all import *
>>> from sage.repl.preparse import preparse_calculus
>>> preparse_calculus(";f(t,s)=t^2;")
';__tmp__=var("t,s"); f = symbolic_expression(t^2).function(t,s);'
>>> preparse_calculus(";f( t , s ) = t^2;")
';__tmp__=var("t,s"); f = symbolic_expression(t^2).function(t,s);'

sage.repl.preparse.preparse_file(contents, globals=None, numeric_literals=True)[source]¶

Preparse contents which is input from a file such as .sage files.

Special attentions are given to numeric literals and load/attach file directives.

Note

Temporarily, if @parallel is in the input, then numeric_literals is always set to False.

INPUT:

contents – string
globals – dictionary or None (default: None); if given, then arguments to load/attach are evaluated in the namespace of this dictionary
numeric_literals – boolean (default: True); whether to factor out wrapping of integers and floats, so they do not get created repeatedly inside loops

OUTPUT: string

sage.repl.preparse.preparse_file_named(name)[source]¶

Preparse file named name (presumably a .sage file), outputting to a temporary file.

This returns the temporary file as a Path object.

EXAMPLES:

Sage

sage: from sage.repl.preparse import preparse_file_named
sage: tmpf = tmp_filename(ext='.sage')
sage: with open(tmpf, 'w') as f:
....:     out = f.write("a = 2")
sage: preparse_file_named(tmpf)
PosixPath('...sage.py')

Python

>>> from sage.all import *
>>> from sage.repl.preparse import preparse_file_named
>>> tmpf = tmp_filename(ext='.sage')
>>> with open(tmpf, 'w') as f:
...     out = f.write("a = 2")
>>> preparse_file_named(tmpf)
PosixPath('...sage.py')

sage.repl.preparse.preparse_file_named_to_stream(name, out)[source]¶: Preparse file named code{name} (presumably a .sage file), outputting to stream code{out}.

sage.repl.preparse.preparse_generators(code)[source]¶

Parse generator syntax, converting:

obj.<gen0,gen1,...,genN> = objConstructor(...)

into:

obj = objConstructor(..., names=("gen0", "gen1", ..., "genN"))
(gen0, gen1, ..., genN,) = obj.gens()

and:

obj.<gen0,gen1,...,genN> = R[interior]

into:

obj = R[interior]; (gen0, gen1, ..., genN,) = obj.gens()

INPUT:

code – string

OUTPUT: string

LIMITATIONS:

The entire constructor must be on one line.

AUTHORS:

2006-04-14: Joe Wetherell (jlwether@alum.mit.edu)
- Initial version.
2006-04-17: William Stein
- Improvements to allow multiple statements.
2006-05-01: William
- Fix bug that Joe found.
2006-10-31: William
- Fix so obj does not have to be mutated.
2009-01-27: Robert Bradshaw
- Rewrite using regular expressions

sage.repl.preparse.preparse_numeric_literals(code, extract=False, quotes="'")[source]¶

Preparse numerical literals into their Sage counterparts, e.g. Integer, RealNumber, and ComplexNumber.

INPUT:

code – string; a code block to preparse
extract – boolean (default: False); whether to create names for the literals and return a dictionary of name-construction pairs
quotes – string (default: "'"); used to surround string arguments to RealNumber and ComplexNumber. If None, will rebuild the string using a list of its Unicode code-points.

OUTPUT:

a string or (string, string:string dictionary) 2-tuple; the preparsed block and, if extract is True, the name-construction mapping

EXAMPLES:

Sage

sage: from sage.repl.preparse import preparse_numeric_literals
sage: preparse_numeric_literals("5")
'Integer(5)'
sage: preparse_numeric_literals("5j")
"ComplexNumber(0, '5')"
sage: preparse_numeric_literals("5jr")
'5J'
sage: preparse_numeric_literals("5l")
'5'
sage: preparse_numeric_literals("5L")
'5'
sage: preparse_numeric_literals("1.5")
"RealNumber('1.5')"
sage: preparse_numeric_literals("1.5j")
"ComplexNumber(0, '1.5')"
sage: preparse_numeric_literals(".5j")
"ComplexNumber(0, '.5')"
sage: preparse_numeric_literals("5e9j")
"ComplexNumber(0, '5e9')"
sage: preparse_numeric_literals("5.")
"RealNumber('5.')"
sage: preparse_numeric_literals("5.j")
"ComplexNumber(0, '5.')"
sage: preparse_numeric_literals("5.foo()")
'Integer(5).foo()'
sage: preparse_numeric_literals("5.5.foo()")
"RealNumber('5.5').foo()"
sage: preparse_numeric_literals("5.5j.foo()")
"ComplexNumber(0, '5.5').foo()"
sage: preparse_numeric_literals("5j.foo()")
"ComplexNumber(0, '5').foo()"
sage: preparse_numeric_literals("1.exp()")
'Integer(1).exp()'
sage: preparse_numeric_literals("1e+10")
"RealNumber('1e+10')"
sage: preparse_numeric_literals("0x0af")
'Integer(0x0af)'
sage: preparse_numeric_literals("0x10.sqrt()")
'Integer(0x10).sqrt()'
sage: preparse_numeric_literals('0o100')
'Integer(0o100)'
sage: preparse_numeric_literals('0b111001')
'Integer(0b111001)'
sage: preparse_numeric_literals('0xe')
'Integer(0xe)'
sage: preparse_numeric_literals('0xEAR')
'0xEA'
sage: preparse_numeric_literals('0x1012Fae')
'Integer(0x1012Fae)'
sage: preparse_numeric_literals('042')
'Integer(42)'
sage: preparse_numeric_literals('000042')
'Integer(42)'

Python

>>> from sage.all import *
>>> from sage.repl.preparse import preparse_numeric_literals
>>> preparse_numeric_literals("5")
'Integer(5)'
>>> preparse_numeric_literals("5j")
"ComplexNumber(0, '5')"
>>> preparse_numeric_literals("5jr")
'5J'
>>> preparse_numeric_literals("5l")
'5'
>>> preparse_numeric_literals("5L")
'5'
>>> preparse_numeric_literals("1.5")
"RealNumber('1.5')"
>>> preparse_numeric_literals("1.5j")
"ComplexNumber(0, '1.5')"
>>> preparse_numeric_literals(".5j")
"ComplexNumber(0, '.5')"
>>> preparse_numeric_literals("5e9j")
"ComplexNumber(0, '5e9')"
>>> preparse_numeric_literals("5.")
"RealNumber('5.')"
>>> preparse_numeric_literals("5.j")
"ComplexNumber(0, '5.')"
>>> preparse_numeric_literals("5.foo()")
'Integer(5).foo()'
>>> preparse_numeric_literals("5.5.foo()")
"RealNumber('5.5').foo()"
>>> preparse_numeric_literals("5.5j.foo()")
"ComplexNumber(0, '5.5').foo()"
>>> preparse_numeric_literals("5j.foo()")
"ComplexNumber(0, '5').foo()"
>>> preparse_numeric_literals("1.exp()")
'Integer(1).exp()'
>>> preparse_numeric_literals("1e+10")
"RealNumber('1e+10')"
>>> preparse_numeric_literals("0x0af")
'Integer(0x0af)'
>>> preparse_numeric_literals("0x10.sqrt()")
'Integer(0x10).sqrt()'
>>> preparse_numeric_literals('0o100')
'Integer(0o100)'
>>> preparse_numeric_literals('0b111001')
'Integer(0b111001)'
>>> preparse_numeric_literals('0xe')
'Integer(0xe)'
>>> preparse_numeric_literals('0xEAR')
'0xEA'
>>> preparse_numeric_literals('0x1012Fae')
'Integer(0x1012Fae)'
>>> preparse_numeric_literals('042')
'Integer(42)'
>>> preparse_numeric_literals('000042')
'Integer(42)'

Test underscores as digit separators (PEP 515, https://www.python.org/dev/peps/pep-0515/):

Sage

sage: preparse_numeric_literals('123_456')
'Integer(123_456)'
sage: preparse_numeric_literals('123_456.78_9_0')
"RealNumber('123_456.78_9_0')"
sage: preparse_numeric_literals('0b11_011')
'Integer(0b11_011)'
sage: preparse_numeric_literals('0o76_321')
'Integer(0o76_321)'
sage: preparse_numeric_literals('0xaa_aaa')
'Integer(0xaa_aaa)'
sage: preparse_numeric_literals('1_3.2_5e-2_2')
"RealNumber('1_3.2_5e-2_2')"

sage: for f in ["1_1.", "11_2.", "1.1_1", "1_1.1_1", ".1_1", ".1_1e1_1", ".1e1_1",
....:           "1e12_3", "1_1e1_1", "1.1_3e1_2", "1_1e1_1", "1e1", "1.e1_1",
....:           "1.0", "1_1.0"]:
....:     preparse_numeric_literals(f)
....:     assert preparse(f) == preparse_numeric_literals(f), f
"RealNumber('1_1.')"
"RealNumber('11_2.')"
"RealNumber('1.1_1')"
"RealNumber('1_1.1_1')"
"RealNumber('.1_1')"
"RealNumber('.1_1e1_1')"
"RealNumber('.1e1_1')"
"RealNumber('1e12_3')"
"RealNumber('1_1e1_1')"
"RealNumber('1.1_3e1_2')"
"RealNumber('1_1e1_1')"
"RealNumber('1e1')"
"RealNumber('1.e1_1')"
"RealNumber('1.0')"
"RealNumber('1_1.0')"

Python

>>> from sage.all import *
>>> preparse_numeric_literals('123_456')
'Integer(123_456)'
>>> preparse_numeric_literals('123_456.78_9_0')
"RealNumber('123_456.78_9_0')"
>>> preparse_numeric_literals('0b11_011')
'Integer(0b11_011)'
>>> preparse_numeric_literals('0o76_321')
'Integer(0o76_321)'
>>> preparse_numeric_literals('0xaa_aaa')
'Integer(0xaa_aaa)'
>>> preparse_numeric_literals('1_3.2_5e-2_2')
"RealNumber('1_3.2_5e-2_2')"

>>> for f in ["1_1.", "11_2.", "1.1_1", "1_1.1_1", ".1_1", ".1_1e1_1", ".1e1_1",
...           "1e12_3", "1_1e1_1", "1.1_3e1_2", "1_1e1_1", "1e1", "1.e1_1",
...           "1.0", "1_1.0"]:
...     preparse_numeric_literals(f)
...     assert preparse(f) == preparse_numeric_literals(f), f
"RealNumber('1_1.')"
"RealNumber('11_2.')"
"RealNumber('1.1_1')"
"RealNumber('1_1.1_1')"
"RealNumber('.1_1')"
"RealNumber('.1_1e1_1')"
"RealNumber('.1e1_1')"
"RealNumber('1e12_3')"
"RealNumber('1_1e1_1')"
"RealNumber('1.1_3e1_2')"
"RealNumber('1_1e1_1')"
"RealNumber('1e1')"
"RealNumber('1.e1_1')"
"RealNumber('1.0')"
"RealNumber('1_1.0')"

Having consecutive underscores is not valid Python syntax, so it is not preparsed, and similarly with a trailing underscore:

Sage

sage: preparse_numeric_literals('123__45')
'123__45'
sage: 123__45
Traceback (most recent call last):
...
SyntaxError: invalid ...

sage: preparse_numeric_literals('3040_1_')
'3040_1_'
sage: 3040_1_
Traceback (most recent call last):
...
SyntaxError: invalid ...

Python

>>> from sage.all import *
>>> preparse_numeric_literals('123__45')
'123__45'
>>> 123__45
Traceback (most recent call last):
...
SyntaxError: invalid ...

>>> preparse_numeric_literals('3040_1_')
'3040_1_'
>>> 3040_1_
Traceback (most recent call last):
...
SyntaxError: invalid ...

Using the quotes parameter:

Sage

sage: preparse_numeric_literals('5j', quotes='"')
'ComplexNumber(0, "5")'
sage: preparse_numeric_literals('3.14', quotes="'''")
"RealNumber('''3.14''')"
sage: preparse_numeric_literals('3.14', quotes=None)
'RealNumber(str().join(map(chr, [51, 46, 49, 52])))'
sage: preparse_numeric_literals('5j', quotes=None)
'ComplexNumber(0, str().join(map(chr, [53])))'

Python

>>> from sage.all import *
>>> preparse_numeric_literals('5j', quotes='"')
'ComplexNumber(0, "5")'
>>> preparse_numeric_literals('3.14', quotes="'''")
"RealNumber('''3.14''')"
>>> preparse_numeric_literals('3.14', quotes=None)
'RealNumber(str().join(map(chr, [51, 46, 49, 52])))'
>>> preparse_numeric_literals('5j', quotes=None)
'ComplexNumber(0, str().join(map(chr, [53])))'

sage.repl.preparse.strip_prompts(line)[source]¶

Remove leading sage: and >>> prompts so that pasting of examples from the documentation works.

INPUT:

line – string to process

OUTPUT: string stripped of leading prompts

EXAMPLES:

Sage

sage: from sage.repl.preparse import strip_prompts
sage: strip_prompts("sage: 2 + 2")
'2 + 2'
sage: strip_prompts(">>>   3 + 2")
'3 + 2'
sage: strip_prompts("  2 + 4")
'  2 + 4'

Python

>>> from sage.all import *
>>> from sage.repl.preparse import strip_prompts
>>> strip_prompts("sage: 2 + 2")
'2 + 2'
>>> strip_prompts(">>>   3 + 2")
'3 + 2'
>>> strip_prompts("  2 + 4")
'  2 + 4'

sage.repl.preparse.strip_string_literals(code, state=None)[source]¶

Return a string with all literal quotes replaced with labels and a dictionary of labels for re-substitution.

This makes parsing easier.

INPUT:

code – string; the input
state – a QuoteStack (default: None); state with which to continue processing, e.g., across multiple calls to this function

OUTPUT:

a 3-tuple of the processed code, the dictionary of labels, and any accumulated state

EXAMPLES:

Sage

sage: from sage.repl.preparse import strip_string_literals
sage: s, literals, state = strip_string_literals(r'''['a', "b", 'c', "d\""]''')
sage: s
'[%(L1)s, %(L2)s, %(L3)s, %(L4)s]'
sage: literals
{'L1': "'a'", 'L2': '"b"', 'L3': "'c'", 'L4': '"d\\""'}
sage: print(s % literals)
['a', "b", 'c', "d\""]
sage: print(strip_string_literals(r'-"\\\""-"\\"-')[0])
-%(L1)s-%(L2)s-

Python

>>> from sage.all import *
>>> from sage.repl.preparse import strip_string_literals
>>> s, literals, state = strip_string_literals(r'''['a', "b", 'c', "d\""]''')
>>> s
'[%(L1)s, %(L2)s, %(L3)s, %(L4)s]'
>>> literals
{'L1': "'a'", 'L2': '"b"', 'L3': "'c'", 'L4': '"d\\""'}
>>> print(s % literals)
['a', "b", 'c', "d\""]
>>> print(strip_string_literals(r'-"\\\""-"\\"-')[Integer(0)])
-%(L1)s-%(L2)s-

Triple-quotes are handled as well:

Sage

sage: s, literals, state = strip_string_literals("[a, '''b''', c, '']")
sage: s
'[a, %(L1)s, c, %(L2)s]'
sage: print(s % literals)
[a, '''b''', c, '']

Python

>>> from sage.all import *
>>> s, literals, state = strip_string_literals("[a, '''b''', c, '']")
>>> s
'[a, %(L1)s, c, %(L2)s]'
>>> print(s % literals)
[a, '''b''', c, '']

Comments are substitute too:

Sage

sage: s, literals, state = strip_string_literals("code '#' # ccc 't'"); s
'code %(L1)s #%(L2)s'
sage: s % literals
"code '#' # ccc 't'"

Python

>>> from sage.all import *
>>> s, literals, state = strip_string_literals("code '#' # ccc 't'"); s
'code %(L1)s #%(L2)s'
>>> s % literals
"code '#' # ccc 't'"

A state is returned so one can break strings across multiple calls to this function:

Sage

sage: s, literals, state = strip_string_literals('s = "some'); s
's = %(L1)s'
sage: s, literals, state = strip_string_literals('thing" * 5', state); s
'%(L1)s * 5'

Python

>>> from sage.all import *
>>> s, literals, state = strip_string_literals('s = "some'); s
's = %(L1)s'
>>> s, literals, state = strip_string_literals('thing" * 5', state); s
'%(L1)s * 5'