[GiNaC-devel] [PATCH] mul: algebraic_subs_mul(), has(): don't write beyond the end of array

[Alexei Sheplyakov]

algebraic_match_mul_with_mul() iterates over operands of mul, that is

for (size_t i=0; i<e.nops(); ++i)

However, the size of arrays (`vectors' in STL speak) passed to this
function is seq.size(), which is nops() - 1 for any mul object. Thus
algebraic_match_mul_with_mul() accesses beyond the arrays limit. Usually
it's not a problem, since any reasonable implementation of std::vector<bool>
packs booleans into ints (or longs). However, some STL implementations
(in particular, the one shipped with msvc) are more picky, and access
beyond the vector<bool> limits results in a segfault. Therefore let's
play safe and allocate proper number of elements (that is, nops()) for
those arrays (subsed and currsubsed).

Thanks to Jan Rheinländer for a bug report.

---
 ginac/mul.cpp |   12 +++++++-----
 1 files changed, 7 insertions(+), 5 deletions(-)

diff --git a/ginac/mul.cpp b/ginac/mul.cpp
index dfda410..3733bc4 100644
--- a/ginac/mul.cpp
+++ b/ginac/mul.cpp
@@ -729,6 +729,9 @@ bool algebraic_match_mul_with_mul(const mul &e, const ex &pat, exmap& repls,
 		int factor, int &nummatches, const std::vector<bool> &subsed,
 		std::vector<bool> &matched)
 {
+	GINAC_ASSERT(subsed.size() == e.nops());
+	GINAC_ASSERT(matched.size() == e.nops());
+
 	if (factor == (int)pat.nops())
 		return true;
 
@@ -760,8 +763,8 @@ bool mul::has(const ex & pattern, unsigned options) const
 	if(is_a<mul>(pattern)) {
 		exmap repls;
 		int nummatches = std::numeric_limits<int>::max();
-		std::vector<bool> subsed(seq.size(), false);
-		std::vector<bool> matched(seq.size(), false);
+		std::vector<bool> subsed(nops(), false);
+		std::vector<bool> matched(nops(), false);
 		if(algebraic_match_mul_with_mul(*this, pattern, repls, 0, nummatches,
 				subsed, matched))
 			return true;
@@ -771,8 +774,7 @@ bool mul::has(const ex & pattern, unsigned options) const
 
 ex mul::algebraic_subs_mul(const exmap & m, unsigned options) const
 {	
-	std::vector<bool> subsed(seq.size(), false);
-	exvector subsresult(seq.size());
+	std::vector<bool> subsed(nops(), false);
 	ex divide_by = 1;
 	ex multiply_by = 1;
 
@@ -781,7 +783,7 @@ ex mul::algebraic_subs_mul(const exmap & m, unsigned options) const
 		if (is_exactly_a<mul>(it->first)) {
 retry1:
 			int nummatches = std::numeric_limits<int>::max();
-			std::vector<bool> currsubsed(seq.size(), false);
+			std::vector<bool> currsubsed(nops(), false);
 			exmap repls;
 			
 			if(!algebraic_match_mul_with_mul(*this, it->first, repls, 0, nummatches, subsed, currsubsed))
-- 
1.7.1