These are the release notes for the independent CAS integration tests
summer 2023 edition
This release is dedicated to the memory of Albert D Rich,
author of the Rubi integration software.
Copy of this release notes in plain text is here
<
https://12000.org/my_notes/CAS_integration_tests/reports/summer_2023/email_release_notes_summer_2023.txt>
Main web page is
<
https://12000.org/my_notes/CAS_integration_tests/reports/summer_2023/index.htm>
Time to complete all tests was 7 months due to having to restart
some tests.
CAS systems used:
------------------
Mathematica 13.3.1 (August 16, 2023) on windows 10.
Rubi 4.16.1 (Dec 19, 2018) on Mathematica 13.3 on windows 10
Maple 2023.1 (July, 12, 2023) on windows 10.
Maxima 5.47 (June 1, 2023) using Lisp SBCL 2.3.0/sagemath 10.1
FriCAS 1.3.9 (July 8, 2023) based on sbcl 2.3.0/sagemath 10.1
Giac/Xcas 1.9.0-57 (June 26, 2023) on Linux/sagemath 10.1
Sympy 1.12 (May 10, 2023) Using Python 3.11.3 on Linux.
Mupad using Matlab 2021a/Symbolic Math Toolbox Version 8.7
Number of integrals
-------------------
sqlite> select COUNT(*) from main;
85963
Number of test input files: [212]
Full database (SQL) is updated which contains all integrals and all
results used to generate all the reports.(1.3 GB). This link has
instructions how to use and install the database.
<
https://12000.org/my_notes/CAS_integration_tests/reports/summer_2023/DATA_BASE/index.htm>
Solved percentage
-------------------
1. Mathematica % 98.233
2. Rubi % 94.217
3. Maple % 86.609
4. Fricas % 80.892
5. Giac % 59.034
6. Maxima(*) % 57.28 --> 57.28+5.516 = % 62.796 (see below on exceptions) 7. Mupad % 57.09
8. Sympy % 43.562
A grade percentage
--------------------
1. Rubi % 87.36
2. Mathematica % 75.34
3. Maple % 60.52
4. Fricas % 49.95
5. Maxima % 39.7
6. Giac % 38.63
7. Sympy % 25.87
8. Mupad N/A (not graded)
Average CPU time per integral. Applied only for passed integrals
(timed out and exception/failed integrals do not count) ---------------------------------------------------------
1. Rubi 0.28 sec.
2. Maxima 0.44 sec.
3. Fricas 1.18 sec.
4. Giac 2.03 sec.
5. Mathematica 2.31 sec.
6. Sympy 5.36 sec.
7. Maple(*) 5.73 sec. (slower than normal as it tries all algorithms).
8. Mupad 6.82 sec.
Average Leaf size
-----------------
No simplification is made to the result. This is the leaf size
of the antiderivative as is.
1. Rubi 154.14
2. Mathematica 311.05
3. Sympy 373.47
4. Maxima 587.33
5. Giac 632.64
6. Mupad 664.7
7. Fricas 1161.58
8. Maple(*) 64603.4 (Maple typically have larger leaf size as it does
not simplify result by default)
Main highlights and changes
----------------------------
1. Added verification to result of Maple.
2. Added number of new statistics diagrams per file and
also for the overall results. Statistics now show
- percentage solved.
- grades.
- CPU time and leaf size.
- Performance based on number of rules Rubi used.
- Performance based on number of steps Rubi used.
- Solved integrals histogram based on leaf size of result.
- Solved integrals histogram based on CPU time used.
- Leaf size vs. CPU time used.
- Performance per integrand type.
- Maximum leaf size ratio for each CAS against the optimal result.
- Pass/Fail per test file for each CAS system.
3. New chapter that lists Integrals which generated an exception
failure F(-2) for each CAS.
Fricas:
-------
sqlite> select DISTINCT fricas_anti from main where fricas_pass=-2;
Error detected within library code: integrate: implementation incomplete (constant residues)
Error detected within library code: integrate: implementation incomplete (has polynomial part)
Error detected within library code: not invertible
Error detected within library code: Curve not irreducible after change of variable 0 -> infinity
Error detected within library code: integrate: implementation incomplete (residue poly has multiple non-linear factors)
Error detected within library code: integrate: implementation incomplete (trace 0)
Error detected within library code: catdef: division by zero
RuntimeError >> no explicit roots found
Error detected within library code: alglogextint: unimplemented
Error detected within library code: Not integrable (provided residues have no relations)
Error detected within library code: Shouldn not happen
Error detected within library code: Too many variables
Error detected within library code: failed of mode Union(SparseUnivariatePolynomial(Expression(Complex(Integer))),failed)
Error detected within library code: failed of mode Union(SparseUnivariatePolynomial(Expression(Integer)),failed)
System error: Heap exhausted 45588576 requested.PROCEED WITH CAUTION. AttributeError
Error detected within library code: integrate: implementation incomplete (irrational residues)
Error detected within library code: do_alg_rde: unimplemented kernel
System error: Heap exhausted (no more space for allocation).PROCEED WITH CAUTION.
Number of problems failed in Fricas due to exception is
sqlite> select COUNT(*) from main where fricas_pass=-2;
2017
Out of total 85963 integrals, this gives 2.34%.
Maxima:
--------
select DISTINCT maxima_anti from main where maxima_pass=-2;
ValueError >> Computation failed since Maxima requested additional constraints; RuntimeError >> ECL says: expt: undefined: 0 to a negative exponent. RuntimeError >> ECL says: THROW: The catch RAT-ERR is undefined.
RuntimeError >> ECL says: In function CAR, the value of the first argument is 1which is not of the expected type LIST
TypeError >> unable to make sense of Maxima expression 'li[2]' in Sage RuntimeError >> ECL says: Memory limit reached. Please jump to an outer pointer, quit program and enlarge thememory limits before executing the program again.
RuntimeError >> ECL says: BINDING-STACK overflow at size 10240. Stack can probably be resized.Proceed with caution.
RuntimeError >> Encountered operator mismatch in maxima-to-sr translation RuntimeError >> ECL says: sign: argument cannot be imaginary; found %i RuntimeError >> ECL says: sign: argument cannot be imaginary; found sqrt((-_SAGE_VAR_d*_SAGE_VAR_x^2)-2)
RuntimeError >> ECL says: sign: argument cannot be imaginary; found sqrt((-_SAGE_VAR_x^2)-2)
RuntimeError >> ECL says: sign: argument cannot be imaginary; found sqrt(-_SAGE_VAR_b)
RuntimeError >> ECL says: sign: argument cannot be imaginary; found sqrt((-_SAGE_VAR_b*_SAGE_VAR_x^2)-_SAGE_VAR_a)
RuntimeError >> ECL says: sign: argument cannot be imaginary; found sqrt(sqrt(5)-5)
The majority of Maxima exceptions are "failed since Maxima requested additional constraints;"
sqlite> select COUNT(*) from main where maxima_pass=-2;
6538
The number of exceptions due to asking for assumptions is
sqlite> select COUNT(*) from main where maxima_pass=-2 and maxima_anti LIKE '%Maxima requested additional constraints%';
4742
Which is 72.53% of all the exceptions!
Out of total 85963 integrals, this gives 5.516%. So in theory, if there
is a way to give Maxima these assumptions in batch mode, and assuming it
can solve the integral after that, its percentage solved can increase by
5.516 % from what it currently obtained.
GIAC:
------
select DISTINCT giac_anti from main where giac_pass=-2;
Invalid _EXT in replace_ext Error: Bad Argument ValueInvalid _EXT in replace_ext Error: Bad Argument ValueDone
The choice was done assuming [sageVARx]=[79]sym2poly/r2sym(const gen & e,const index_m & i,const vecteur & l) Error: B
Limit: Max order reached or unable to make series expansion Error: Bad Argument Value
index.cc index_m i_lex_is_greater Error: Bad Argument Value
Unable to divide, perhaps due to rounding error%%%{1,[0,3,1,0,0]%%%} / %%%{1,[0,0,0,1,1]%%%} Error: Bad Argument Value
The choice was done assuming 0=[0,0]ext_reduce Error: Bad Argument Typeintegrate(1/4*16*i*((sageVARa+(-i)*sageVARa*sageVARx)^(1/4))^10/(-((sa
The choice was done assuming 0=[0,0]ext_reduce Error: Bad Argument Typeintegrate(4*i*((sageVARa+(-i)*sageVARa*sageVARx)^(1/4))^6/(-((sageVARa
int(sage0,sageVARx):;OUTPUT:The choice was done assuming 0=[0,0]ext_reduce Error: Bad Argument TypeDone
ext_reduce Error: Bad Argument Typeintegrate(4*i*((sageVARa+(-i)*sageVARa*sageVARx)^(1/4))^4/(-((sageVARa
Bad Argument Typeintegrate(4*i*((sageVARa+(-i)*sageVARa*sageVARx)^(1/4))^6/((-((sageVAR
Unable to convert to real sageVARb Error: Bad Argument ValueRecursive assumption
AttributeError >> type
ignoredsym2poly/r2sym(const gen & e,const index_m & i,const vecteur & l) Error: Bad Argument Value
index.cc index_m operator + Error: Bad Argument Value
sym2poly/r2sym(const gen & e,const index_m & i,const vecteur & l) Error: Bad Argument Valuesym2poly/r2sym(const gen &
unable to parse Giac output
Francis algorithm failure for[-1.0,infinity,infinity,infinity,infinity]proot error [1.0,infinity,infinity,infinity,inf
Error index.cc index_gcd Error: Bad Argument ValueError index.cc index_gcd Error: Bad Argument ValueDone
Not invertible Error: Bad Argument Value
Polynomial exponent overflow. Error: Bad Argument Value
The choice was done assuming 0=[0,0,0]ext_reduce Error: Bad Argument TypeThe choice was done assuming 0=[0,0,0]ext_red
index_gcd Error: Bad Argument Value
index.cc index_m operator + Error: Bad Argument Valueindex.cc index_m operator + Error: Bad Argument ValueDone
The choice was done assuming 0=[0,0]ext_reduce Error: Bad Argument TypeThe choice was done assuming 0=[0,0]ext_reduce
Non regular value [0] was discarded and replaced randomly by 0=[-38]Warning, replacing -38 by 97, a substitution varia
Unable to find common minimal polynomial Error: Bad Argument ValueDone
Warning, replacing 0 by 15, a substitution variable should perhaps be purged.Warning
Non regular value [0] was discarded and replaced randomly by 0=[18]Warning, replacing 18 by 97, a substitution variabl
index.cc index_m i_lex_is_greater Error: Bad Argument Value
ignoredRecursive assumption
Limit: Max order reached or unable to make series expansion Error: Bad Argument Value
ln of unsigned or minus infinity Error: Bad Argument Value
Invalid series expansion: non tractable function acos at +infinity
Minimal poly. in rootof must be fraction free Error: Bad Argument ValueMinimal poly. in rootof must be fraction free E
sym2poly exception caught Unable to convert to real %%{poly1[1.0000000000000000000000000000000,0.000000000000000000000
Unable to transpose Error: Bad Argument ValueUnable to transpose Error: Bad Argument Valuesym2poly/r2sym(const gen & e
Invalid _EXT in replace_ext Error: Bad Argument Valueintegrate
rootof minimal polynomial must be unitary Error: Bad Argument Valuerootof minimal polynomial must be unitary Error: Ba
Warning: increasing stack size to 2048000. *** Warning: increasing stack size to 2048000. *** Warning: increasing stack size to
proot error [1,0,0,0,1,0,0,0,1]proot error [1,0,0,0,-1,0,0,0,1]proot error [1,0,-10,0,1]proot error [1,0,-10,0,1]proot
index.cc index_gcd Error: Bad Argument ValueError index.cc index_gcd Error: Bad Argument ValueError index.cc ind
Bad Argument Valuesym2poly/r2sym(const gen & e,const index_m & i,con
int(sage0,sageVARx):;OUTPUT:int(sage0,sageVARx) Error: Invalid dimension
ln of unsigned or minus infinity Error: Bad Argument Value
NotImplementedError >> unable to parse Giac output
simplify: Polynomials do not have the same dimension Error: Bad Argument Value
The majority of exceptions are "Unable to divide, perhaps due to rounding error"
Number of problems failed in GIAC due to exception is
sqlite> select COUNT(*) from main where giac_pass=-2;
3341
Out of total 85963 integrals, this gives 3.88%.
Sympy
------
select DISTINCT sympy_anti from main where sympy_pass=-2;
TypeError >> Invalid NaN comparison
RecursionError >> maximum recursion depth exceeded
TypeError >> Invalid comparison of non-real I
HeuristicGCDFailed >> no luck
MellinTransformStripError >> Pole inside critical strip?
NotImplementedError >> no valid subset found
PolynomialError >> 1/(128*_t**4 - 16*sqrt(2)*_t**2 + 1) contains an element of the set of generators.
TypeError >> Invalid comparison of non-real zoo
TypeError >> '>' not supported between instances of 'Poly' and 'int'
AttributeError >> 'NoneType' object has no attribute 'primitive'
TypeError >> Invalid comparison of non-real -I
CoercionFailed >> Cannot convert _t0**2 + exp(2*I*a) of type <class 'sympy.core.add.Add'> to QQ_I[x,b,_t0,exp(I*a)]
TypeError >> cannot determine truth value of Relational
ValueError >> Exceeds the limit (4300 digits) for integer string conversion; use sys.set_int_max_str_digits() to increase the limit
CoercionFailed >> Cannot convert 5*exp(exp(exp(5))) of type <class 'sympy.core.mul.Mul'> to ZZ[exp(exp(exp(5))/2),sqrt(16 + 5*exp(exp(exp(5))))]
GeneratorsError >> polynomial ring and it's ground domain share generators
IndexError >> Index out of range: a[1]
The majority of exceptions are "PolynomialError"
Number of problems failed in sympy due to exception is
sqlite> select COUNT(*) from main where sympy_pass=-2;
823
Out of total 85963 integrals, this gives 0.957%.
The number of exceptions due to "PolynomialError" is
sqlite> select COUNT(*) from main where sympy_pass=-2 and sympy_anti LIKE '%PolynomialError%';
64
Which is 7.77% of all the sympy exceptions.
4. Listing of integrals solved by CAS which has no known antiderivatives.
Note about CPU timing
----------------------
For Mathematica, Rubi and Mupad 3 CPU minutes were used to complete each integration problem. All on windows 10. For Maple it was given 5 minutes because 'method'=':-_RETURNVERBOSE' was used which makes it
try all available methods.
For Fricas, Giac and Maxima, 5 minutes were also given to compensate
for calling them via sagemath which adds extra time to start
new proccess each time for each problem. Note that the time recorded
for Fricas, Giac and Maxima is inflated by some amount due the time
it takes to start new process each time. About 20-30% of the time used
can be attributed to the overhead of using sagemath to interface to these systems.
Fricas, Giac, Maxima and sympy were all run on Linux inside Virtual
box hosted on windows 10 but with over 50 GB of RAM and on very fast PC.
Maximum anti derivatives generated per CAS ------------------------------------------
Maple:
------
leaf size 10,839
sqlite> select command_in_maple, MAX(LENGTH(maple_anti)) from main;
int(-2^(1/2)*(x^4+2*x^2+4*x+1)^(1/2)+x*(-1+2*2^(1/2)),x,method=_RETURNVERBOSE)|10839
Mathematica:
---------------
leaf size 10,005
sqlite> select command_in_mma, MAX(LENGTH(mma_anti)) from main;
Integrate[((a + b*Sec[c + d*x])^(3/2)*(A + B*Sec[c + d*x] + C*Sec[c + d*x]^2))/Sec[c + d*x]^(7/2),x]|10005
Rubi:
-----
leaf size 41,795
sqlite> select command_in_mma, MAX(LENGTH(rubi_anti)) from main;
Integrate[(E^(30 + 8*x)*(56 + 8*E^10 + E^5*(24 - 16*x) - 24*x + 8*x^2)
+ E^(15 + 4*x)*(-2016 - 8*E^30 + 2208*x - 1760*x^2 + 800*x^3 - 280*x^4
+ 56*x^5 - 8*x^6 + E^25*(-56 + 48*x) + E^20*(-280 + 280*x - 120*x^2)
+ E^15*(-800 + 1120*x - 560*x^2 + 160*x^3) + E^10*(-1760 + 2400*x -
1680*x^2 + 560*x^3 - 120*x^4) + E^5*(-2208 + 3520*x - 2400*x^2 + 1120*x^3
- 280*x^4 + 48*x^5))*Log[3])/(7776 + E^50 + E^45*(10 - 10*x) - 12960*x
+ 15120*x^2 - 11520*x^3 + 6960*x^4 - 3152*x^5 + 1160*x^6 - 320*x^7
+ 70*x^8 - 10*x^9 + x^10 + E^40*(70 - 90*x + 45*x^2) + E^35*(320 - 560*x
+ 360*x^2 - 120*x^3) + E^30*(1160 - 2240*x + 1960*x^2 - 840*x^3 + 210*x^4)
+ E^25*(3152 - 6960*x + 6720*x^2 - 3920*x^3 + 1260*x^4 - 252*x^5)
+ E^20*(6960 - 15760*x + 17400*x^2 - 11200*x^3 + 4900*x^4 - 1260*x^5
+ 210*x^6) + E^15*(11520 - 27840*x + 31520*x^2 - 23200*x^3 + 11200*x^4
- 3920*x^5 + 840*x^6 - 120*x^7) + E^10*(15120 - 34560*x + 41760*x^2
- 31520*x^3 + 17400*x^4 - 6720*x^5 + 1960*x^6 - 360*x^7 + 45*x^8)
+ E^5*(12960 - 30240*x + 34560*x^2 - 27840*x^3 + 15760*x^4 - 6960*x^5
+ 2240*x^6 - 560*x^7 + 90*x^8 - 10*x^9)),x]|41795
Mupad
------
Leaf size 1,099,610
select command_in_mupad, MAX(LENGTH(mupad_anti)) from main;
int(1/((a + b*sin(e + f*x))^3*(c + d*sin(e + f*x))^3),x)|1099610
Fricas
------
Leaf size 25,209
sqlite> select command_in_fricas, MAX(LENGTH(fricas_anti)) from main;
integrate(1/((-1+2*x)^(1/2)*(4+3*x)+(1+x)*(-3+4*x)^(1/2)),x, algorithm="fricas")|25209
Giac
-----
Leaf size 5,171,142
sqlite> select command_in_giac, MAX(LENGTH(giac_anti)) from main;
integrate(cos(b*x+a)^3*sin(b*x+a)^2/(d*x+c)^4,x, algorithm="giac")|5171142
Maxima
-------
Leaf size 3,541,205
sqlite> select command_in_maxima, MAX(LENGTH(maxima_anti)) from main;
integrate(cos(d*x+c)^2*(A+B*cos(d*x+c)+C*cos(d*x+c)^2)/(a+a*cos(d*x+c))^(1/2),x, algorithm="maxima")|3541205
Sympy:
------
Leaf size 1,139,416
sqlite> select command_in_sympy, MAX(LENGTH(sympy_anti)) from main;
integrate(sinh(d*x+c)/(a+b*sinh(d*x+c)**2),x)|1139416
Any bugs or problems or issues, please let me know and I'll
try to fix it.
--Nasser
--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)