Makefile.am update

.gitignore

@@ -77,4 +77,7 @@ doc_questions.md
 instructions
 suspected_bugs.md
 optimize_discussion*.md
+claude-resume
+*.log 
+*.trs
 # CVS default ignores end

CLAUDE.md

@@ -14,11 +14,11 @@ decisions and plans agreed with the author, so that they survive a fresh clone.
 
 ## Build and test
 - Build the library with `make -j 8` (after header changes most files rebuild; run it in the background).
-- Flags used: `-std=gnu++11 -O3 -DDEBUG -DNO_STRASSEN -DFORTRAN_ -DNONCLAPACK`, libs `-lcblas -lblas -llapack`.
+- Flags used: `-std=gnu++11 -O3 -DDEBUG -DFORTRAN_ -DNONCLAPACK`, libs `-lcblas -lblas -llapack`.
-  Quick test program: `g++ -std=gnu++11 -I<repo> -O0 -DDEBUG -DNO_STRASSEN -DFORTRAN_ -DNONCLAPACK X.cc <repo>/.libs/libla.a -lcblas -lblas -llapack`
+  Quick test program: `g++ -std=gnu++11 -I<repo> -O0 -DDEBUG -DFORTRAN_ -DNONCLAPACK X.cc <repo>/.libs/libla.a -lcblas -lblas -llapack`
   (add `-fsanitize=address` for memory checks).
 - Regression test of the optimizers: `make check` builds `testoptimize`; `./testoptimize` must report
-  "123 passed, 0 failed" (exit code = number of failures). New optimizer tests go to testoptimize.cc;
+  "137 passed, 0 failed" (exit code = number of failures). New optimizer tests go to testoptimize.cc;
   test problems live in testoptimize.h.
 - `t` (t.cc) is a collection of if(0)/if(1) test blocks, most read n from stdin (`echo 10 | ./t`).
 - NONCBLAS check: copy the repo with `cp -a` (keeps timestamps, otherwise automake reruns and fails),
@@ -66,6 +66,12 @@ decisions and plans agreed with the author, so that they survive a fresh clone.
   adds x at (i,j) and conj(x) at (j,i); NRSMat(SparseSMat) takes the lower triangle; operator* is the
   general product A B (correct for hermitian, complex symmetric exp(iH) and general matrices);
   SparseSMat::gemm computes A^T B (A^\dagger B with 'c'); otimes_sparse = M + M^\dagger incl. doubled diagonal.
+- CSRMat (completed 2026-11): compressed row format, members NRVec rowptr/col/v (share independently, value-only operations keep
+  the index arrays shared), columns strictly ascending in each row, full matrix always stored; flag symmetric only DECLARES hermitian
+  (A=A^dagger, for davidson/lanczos), set by CSRMat(NRSMat) and for real types from a symmetric SparseMat, never checked. Built by
+  conversion (SparseMat, SparseSMat, NRMat, NRSMat, arrays); conversions drop |x|<=sparse_epsilon, arithmetic keeps cancelled zeros
+  (simplify). dot conjugates the left operand (as NRMat); permuteme_rows permutes only rows; CSR*CSR = Gustavson; types double, float,
+  complex<double>, complex<float>, CPU only (cuSPARSE format for later). Tests bugtests/core/csrmat1,3,4,5 and iterative/csrmat2.
 - SparseMat with the symmetric flag is complex symmetric (no conjugation); its permuteme_* members use
   0-based matrix indices, NRPerm/CyclePerm are always 1-based.
 - Permutations: applypermutation(p,v,inverse) uses the same convention as NRVec::permuted (r_{p(i)}=v_i
@@ -83,17 +89,23 @@ decisions and plans agreed with the author, so that they survive a fresh clone.
 - a-X and a/X (scalar minus/divided by object) are deleted on purpose; a single integer constructor is
   forbidden for NRMat, NRMat_from1, Mat3 (Mat3(double) works).
 - Real-storage gdiagonalize uses the dgeev convention (Re + i Im).
-- The NRMat offset view constructor is a feature. MATPTR is kept (rarely used). CSRMat is unfinished and
+- The NRMat offset view constructor is a feature. MATPTR is kept (rarely used). CSRMat: see above. NRMat::strassen (2026-11-06): native Strassen-Winograd in strassen.cc, gemm-compatible
-  documented as unsupported, but kept. Strassen is experimental. CUDA (CUDALA) is deferred/experimental
+  parameters + cutoff (default LA_traits<T>::strassen_cutoff()=2048 all types, to be tuned on HPC), explicit only (gemm stays classical),
+  CBLAS leaves on strided quadrants, device primitives separated for CUDA; NO_STRASSEN removed everywhere. CUDA (CUDALA) is deferred/experimental
   until the author has NVIDIA RTX hardware (known CUDA issues collected in suspected_bugs.md F10).
 - The regression module (reg.*, regsurf.*) is experimental, not much tested in practice (remarked in the manual);
   its NEA oscillator strength f=dE_min/dE is a placeholder for the true one.
 
 ## Planned (only when the author asks)
-- float version: float and complex<float> must ALWAYS use BLAS/LAPACK (s*/c* routines, noncblas wrappers)
+- float version (IN PROGRESS since 2026-11-05): float and complex<float> must ALWAYS use BLAS/LAPACK (s*/c* routines,
-  like double/complex<double>, never generic loops. INSTANTIZE(float) in optimize.cc currently needs float
+  noncblas wrappers) like double/complex<double>, never generic loops. Done: noncblas, vec, mat, smat, nonclass (+ complexify
-  versions of diagonalize, linear_solve, singular_decomposition. BLAS-based NRVec members are meant for
+  BLAS for NRSMat/NRMat), sparsemat, sparsesmat, tensor, fourindex, polynomial, header templates (matexp,
-  floating types only, not integers.
+  conjgrad, gmres, davidson, lanczos, diis: no changes needed beyond the defaults). Float twins are generated from the double specializations (placed right after them). Precision-
+  dependent defaults live in LA_precision<T> (la_traits.h, only double and float; double values unchanged). optimize done
+  2026-11-05: own trait opt_precision<T> in optimize.h, INSTANTIZE(float), float tests in testoptimize.cc (test_float);
+  float LM on badly scaled problems needs my_scaling=true. reg/regsurf, CUDA stay double-only.
+  Tensor Tucker bugs fixed 2026-11-06 (complex: factor = vt.transpose(false); unnamed tensors with inverseorder; bugtests/tensor/tucker.cc).
+  BLAS-based NRVec members are meant for floating types only, not integers.
 - NRSMat: possible bool member `hermitian` toggling hermitian vs complex symmetric behaviour
   (Tensor(const NRSMat&) must then respect it).
 - SparseSMat: possible member `symmetry_type` (0 none, 1 symmetric, 2 hermitian) for NRSMat conversion,
@@ -112,7 +124,9 @@ decisions and plans agreed with the author, so that they survive a fresh clone.
   regression D1-D16, iterative A26-A30 + A38 (2026-10-29), misc C31, C38-C42, B4 (2026-10-30): all bugs closed;
   all doc questions answered (2026-10-30), the manual has no open question boxes.
 - configure.ac and Makefile.am were changed 2026-10-30 (--enable-fortran64int, TESTS = testoptimize, targets manual/doc,
-  doxygen, all-local builds both if pdflatex/doxygen found; MANUAL_TEX lists the .tex files - add new chapters there): run autoreconf.
+  doxygen; all-local builds only the manual (if pdflatex found), doxygen only on request; MANUAL_TEX lists the .tex files - add new chapters there): run autoreconf.
+- configure.ac (2026-11-06): OpenBLAS (with LAPACK+CBLAS) preferred after MKL, only -lopenblas linked; --without-openblas;
+  not with --enable-fortran64int (its parsing moved up). Test programs then link with -lopenblas instead of -lcblas -lblas -llapack.
 - matexp.h: exp()/sincos() default horner=false (polynom, faster for dense); use horner=true for sparse matrices.
 - AuxStorage temporaries go to $TMPDIR (else cwd) and are unlinked at once; AuxStorage and DIIS are not copyable.
 - reg.cc/regsurf.cc cache raw pointers into their NRVec/NRMat members (set in Init): never reallocate those

Makefile.am

@@ -30,7 +30,7 @@ AM_CXXFLAGS=
 
 AM_CXXFLAGS += -g 
 AM_CXXFLAGS += $(OPTIMIZEOPT) $(CUDAOPT) $(FORINTOPT) $(DEBUGOPT) $(PROFILEOPT) $(MATPTROPT)
-AM_CXXFLAGS += -DNO_STRASSEN -DFORTRAN_
+AM_CXXFLAGS += -DFORTRAN_
 
 AM_CXXFLAGS += $(CBLASOPT) $(CLAPACKOPT) 
 AM_CXXFLAGS += $(MKLOPT)
@@ -52,9 +52,9 @@ version.cc: always
 #include $(top_srcdir)/aminclude.am
 
 #documentation: make manual (alias doc) builds doc/manual.pdf, make doxygen the doxygen pages in doxygen-doc,
-#make all builds both besides the library (each only if its tool was found by configure)
+#make all builds manual besides the library (each only if its tool was found by configure)
 MANUAL_TEX = doc/biblio.tex doc/bitvector.tex doc/building.tex doc/concepts.tex doc/fourindex.tex doc/introduction.tex doc/iterative.tex doc/lamacros.tex doc/manual.tex doc/mat.tex doc/nonclass.tex doc/optimize.tex doc/permutation.tex doc/polynomial.tex doc/quaternion.tex doc/regression.tex doc/smat.tex doc/sparse.tex doc/tensor.tex doc/utilities.tex doc/vec.tex doc/vecmat3.tex \
-	doc/bib/iterative.tex doc/bib/optimize.tex doc/bib/permutation.tex doc/bib/polynomial.tex doc/bib/quaternion.tex doc/bib/utilities.tex doc/bib/vecmat3.tex
+	doc/bib/iterative.tex doc/bib/mat.tex doc/bib/optimize.tex doc/bib/permutation.tex doc/bib/polynomial.tex doc/bib/quaternion.tex doc/bib/sparse.tex doc/bib/utilities.tex doc/bib/vecmat3.tex
 EXTRA_DIST += doc/manual.pdf $(MANUAL_TEX)
 
 doc/manual.pdf: $(MANUAL_TEX)

README

@@ -69,8 +69,11 @@ USING:
 NOTE: 
 We named the classes NRMat, NRVec etc. since we started to use them together with the Numerical Recipes library. They can be used with Numerical Recipes in C++, however, they are completely independent of this library.
 
-This is a beta release. The library has already been in use, but not every function/method has been tested.
+The library has already been in use, but not every function/method has been tested.
-There is no documentation except the source itself, since I originaly did not intend to distribute it.
+Recently, I used AI for code review, bugfixes, and generating documentation.
+
+The repository is from now on being mirrored at github, while the primary server remains git.pittnerovi.com.
+
 I will appreciate your bugfixes, exhancements, suggestions etc.
 
 Jiri Pittner

configure.ac

@@ -40,14 +40,41 @@ then
 	MKLOPT="-DHAS_MKL"
 	AC_SUBST([MKLOPT])
 else
-	echo "MKL LIBRARY OR HEADER NOT FOUND, will try ATLAS next!"
+	echo "MKL LIBRARY OR HEADER NOT FOUND, will try OpenBLAS next!"
+fi
+
+FORINTOPT=""
+forint64=""
+AC_ARG_ENABLE([fortran64int],[  --enable-fortran64int to link with 64-bit-integer-BLAS+LAPACK ],[forint64=${enableval}],)
+AC_ARG_ENABLE([fotran64int],[  --enable-fotran64int old misspelled alias of --enable-fortran64int ],[forint64=${enableval}],)
+
+# OpenBLAS (with LAPACK and CBLAS included) is preferred if MKL is not present; not for 64-bit integers
+openblas_present=0
+AC_ARG_WITH([openblas],[  --without-openblas  do not use OpenBLAS even if found (use ATLAS or reference BLAS)],[],[with_openblas=yes])
+case "${forint64}" in
+	""|no) ;;
+	*) with_openblas=no ;;
+esac
+if test $mkl_present == 0 -a "${with_openblas}" != no
+then
+AC_CHECK_LIB([openblas], [dgeev_], [AC_CHECK_LIB([openblas], [cblas_dgemm], [openblas_present=1])])
+fi
+if test $openblas_present == 1
+then
+	echo "OPENBLAS FOUND"
+	BLASLIB="-lopenblas"
+	AC_SUBST([BLASLIB])
+	CLAPACKOPT=-DNONCLAPACK
+	AC_SUBST([CLAPACKOPT])
+else
+	echo "OPENBLAS NOT FOUND OR NOT USED, will try ATLAS next!"
 fi
 
 
 # Checks for mandatory libraries.
 ATLASLIB=""	
 
-if test $mkl_present == 0
+if test $mkl_present == 0 -a $openblas_present == 0
 then
 AC_CHECK_LIB([atlas], [ATL_zgemv],ATLASLIB="-lblas -latlas", [
         echo "ATLAS not found, I hope you are using some other efficient BLAS!"
@@ -75,7 +102,13 @@ AC_ARG_ENABLE([matptr],[  --enable-matptr   switch to double** matrix representa
 CBLASLIB=""
 CBLASOPT=""
 
-if test $mkl_present == 0
+if test $openblas_present == 1
+then
+AC_CHECK_HEADER([cblas.h],,[CBLASOPT=-DNONCBLAS], AC_INCLUDES_DEFAULT)
+AC_SUBST([CBLASOPT])
+fi
+
+if test $mkl_present == 0 -a $openblas_present == 0
 then
 AC_CHECK_LIB([cblas], [cblas_ddot], [CBLASLIB="-lcblas"], [CBLASOPT=-DNONCBLAS],[$ATLASLIB])
 AC_CHECK_HEADER([cblas.h],,[CBLASOPT=-DNONCBLAS CBLASLIB=""], AC_INCLUDES_DEFAULT)
@@ -107,10 +140,6 @@ AC_SUBST([MATPTROPT])
 
 
 #process options
-FORINTOPT=""
-forint64=""
-AC_ARG_ENABLE([fortran64int],[  --enable-fortran64int to link with 64-bit-integer-BLAS+LAPACK ],[forint64=${enableval}],)
-AC_ARG_ENABLE([fotran64int],[  --enable-fotran64int old misspelled alias of --enable-fortran64int ],[forint64=${enableval}],)
 if test -n "${forint64}"; then
     case "${forint64}" in
      yes) FORINTOPT="-DLONG_FORTRAN_INT" 

conjgrad.h

@@ -34,7 +34,7 @@ namespace LA {
 
 
 template<typename T, typename Matrix>
-extern bool conjgrad(const Matrix &bigmat, const NRVec<T> &b, NRVec<T> &x, const bool doguess=true, const double tol=1e-8, const int itmax=1000, const bool verbose=true, bool issquare=1,const bool precondition=1)
+extern bool conjgrad(const Matrix &bigmat, const NRVec<T> &b, NRVec<T> &x, const bool doguess=true, const double tol=LA_precision<typename LA_traits<T>::normtype>::conjgrad_tol(), const int itmax=1000, const bool verbose=true, bool issquare=1,const bool precondition=1)
 {
 int m=bigmat.nrows();
 int n=bigmat.ncols();

csrmat.cc

@@ -1,6 +1,6 @@
 /*
     LA: linear algebra C++ interface library
-    Copyright (C) 2008 Jiri Pittner <jiri.pittner@jh-inst.cas.cz> or <jiri@pittnerovi.com>
+    Copyright (C) 2008-2026 Jiri Pittner <jiri.pittner@jh-inst.cas.cz> or <jiri@pittnerovi.com>
 
     This program is free software: you can redistribute it and/or modify
     it under the terms of the GNU General Public License as published by
@@ -24,38 +24,955 @@
 #include <fcntl.h>
 #include <errno.h>
 #include "csrmat.h"
+#include "nonclass.h"
+#include <algorithm>
 
 namespace LA {
 
+//raw pointers of possibly empty arrays (NULL when empty)
+template<typename U> static inline const U *s_ptr(const NRVec<U> &x) {return x.size()? (const U *)x : NULL;}
+template<typename U> static inline U *s_wptr(NRVec<U> &x) {return x.size()? (U *)x : NULL;}
+
+template <typename T>
+CSRMat<T>::CSRMat(const SPMatindex n, const SPMatindex m)
+: nn(n), mm(m), symmetric(false), rowptr(n+1)
+{
+rowptr.clear();
+}
+
+template <typename T>
+CSRMat<T>::CSRMat(const SPMatindex n)
+: nn(n), mm(n), symmetric(false), rowptr(n+1)
+{
+rowptr.clear();
+}
+
+template <typename T>
+CSRMat<T>::CSRMat(const SPMatindex n, const SPMatindex m, const NRVec<SPMatindex> &rp, const NRVec<SPMatindex> &c, const NRVec<T> &x)
+: nn(n), mm(m), symmetric(false), rowptr(rp), col(c), v(x)
+{
+#ifdef DEBUG
+if(rp.size()!=(int)n+1 || rp[0]!=0 || (int)rp[n]!=c.size() || c.size()!=x.size()) laerror("inconsistent arrays in CSRMat constructor");
+for(SPMatindex i=0; i<n; ++i)
+	{
+	if(rp[i+1]<rp[i]) laerror("decreasing rowptr in CSRMat constructor");
+	for(SPMatindex k=rp[i]; k<rp[i+1]; ++k) if(c[k]>=m || (k>rp[i] && c[k]<=c[k-1])) laerror("column index out of range or columns not ascending in CSRMat constructor");
+	}
+#endif
+}
+
+template <typename T>
+void CSRMat<T>::copyonwrite(bool detachonly, bool deep)
+{
+if(rowptr.size()) rowptr.copyonwrite(detachonly);
+if(col.size()) col.copyonwrite(detachonly);
+if(v.size()) v.copyonwrite(detachonly,deep);
+}
+
+template <typename T>
+void CSRMat<T>::resize(const SPMatindex n, const SPMatindex m)
+{
+nn=n; mm=m; symmetric=false;
+col.resize(0); v.resize(0);
+if(n==0 && m==0) {rowptr.resize(0); return;}
+rowptr.resize(n+1);
+rowptr.clear();
+}
+
+//counting sort, O(nnz+n+m); the columns of the result are ascending in each row also if they were not in the input
+template <typename T>
+const CSRMat<T> CSRMat<T>::transpose(bool conj) const
+{
+CSRMat<T> r(mm,nn);
+r.symmetric=symmetric;
+const int nz=v.size();
+if(nz==0) return r;
+const SPMatindex *rp=s_ptr(rowptr), *c=s_ptr(col);
+const T *x=s_ptr(v);
+r.col.resize(nz); r.v.resize(nz);
+SPMatindex *rrp=r.rowptr, *rc=r.col;
+T *rv=r.v;
+for(int k=0; k<nz; ++k) ++rrp[c[k]+1];
+for(SPMatindex j=0; j<mm; ++j) rrp[j+1]+=rrp[j];
+for(SPMatindex i=0; i<nn; ++i) for(SPMatindex k=rp[i]; k<rp[i+1]; ++k)
+	{
+	SPMatindex p=rrp[c[k]]++;
+	rc[p]=i;
+	rv[p]= conj? LA_traits<T>::conjugate(x[k]) : x[k];
+	}
+for(SPMatindex j=mm; j>0; --j) rrp[j]=rrp[j-1]; //rrp[j] was advanced to the start of row j+1
+rrp[0]=0;
+return r;
+}
+
+//removes small elements; also sums duplicates (adjacent in sorted rows, only during the construction from coordinates)
+template <typename T>
+void CSRMat<T>::simplify(const typename LA_traits<T>::normtype eps)
+{
+const int nz=v.size();
+if(nz==0) return;
+copyonwrite();
+SPMatindex *rp=rowptr, *c=col;
+T *x=v;
+int p=0, k=0;
+for(SPMatindex i=0; i<nn; ++i)
+	{
+	const int end=rp[i+1];
+	while(k<end)
+		{
+		const SPMatindex cc=c[k];
+		T s=x[k++];
+		while(k<end && c[k]==cc) s+=x[k++];
+		if(std::abs(s)>eps) {c[p]=cc; x[p]=s; ++p;}
+		}
+	rp[i+1]=p;
+	}
+if(p<nz) {col.resize(p,true); v.resize(p,true);}
+}
+
+//from coordinate format: grouped by columns (i.e. the transposed matrix in CSR, rows unsorted), transposed back (rows sorted), duplicates summed
+template <typename T>
+void CSRMat<T>::fromcoo(const SPMatindex n, const SPMatindex m, const int nz, const SPMatindex *ri, const SPMatindex *ci, const T *vi)
+{
+CSRMat<T> t(m,n);
+if(nz)
+	{
+	t.col.resize(nz); t.v.resize(nz);
+	SPMatindex *trp=t.rowptr, *tc=t.col;
+	T *tv=t.v;
+	for(int k=0; k<nz; ++k)
+		{
+#ifdef DEBUG
+		if(ri[k]>=n || ci[k]>=m) laerror("index out of range in CSRMat construction");
+#endif
+		++trp[ci[k]+1];
+		}
+	for(SPMatindex j=0; j<m; ++j) trp[j+1]+=trp[j];
+	for(int k=0; k<nz; ++k) {SPMatindex p=trp[ci[k]]++; tc[p]=ri[k]; tv[p]=vi[k];}
+	for(SPMatindex j=m; j>0; --j) trp[j]=trp[j-1];
+	trp[0]=0;
+	}
+*this=t.transpose();
+simplify();
+}
+
+//duplicates summed; a symmetric SparseMat (one triangle, complex symmetric) is unfolded without conjugation
+template <typename T>
+CSRMat<T>::CSRMat(const SparseMat<T> &rhs)
+: nn(0), mm(0), symmetric(false)
+{
+const bool sym=rhs.issymmetric();
+int nz=0;
+for(const matel<T> *l=rhs.getlist(); l; l=l->next) nz+= (sym && l->row!=l->col)? 2:1;
+NRVec<SPMatindex> ri(nz), ci(nz);
+NRVec<T> vi(nz);
+SPMatindex *r=s_wptr(ri), *c=s_wptr(ci);
+T *x=s_wptr(vi);
+int p=0;
+for(const matel<T> *l=rhs.getlist(); l; l=l->next)
+	{
+	r[p]=l->row; c[p]=l->col; x[p++]=l->elem;
+	if(sym && l->row!=l->col) {r[p]=l->col; c[p]=l->row; x[p++]=l->elem;}
+	}
+fromcoo(rhs.nrows(),rhs.ncols(),nz,r,c,x);
+symmetric= sym && !LA_traits<T>::is_complex();
+}
+
+template <typename T>
+CSRMat<T>::CSRMat(const SparseSMat<T> &rhs)
+: nn(rhs.nrows()), mm(rhs.ncols()), symmetric(false), rowptr(nn+1)
+{
+const typename LA_traits<T>::normtype eps=LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon();
+typedef typename std::map<SPMatindex,T>::const_iterator IT;
+int nz=0;
+for(SPMatindex i=0; i<nn; ++i) {const std::map<SPMatindex,T> *l=rhs.line(i); if(l) for(IT q=l->begin(); q!=l->end(); ++q) if(std::abs(q->second)>eps) ++nz;}
+col.resize(nz); v.resize(nz);
+SPMatindex *rp=rowptr, *c=s_wptr(col);
+T *x=s_wptr(v);
+int p=0;
+rp[0]=0;
+for(SPMatindex i=0; i<nn; ++i)
+	{
+	const std::map<SPMatindex,T> *l=rhs.line(i);
+	if(l) for(IT q=l->begin(); q!=l->end(); ++q) if(std::abs(q->second)>eps) {c[p]=q->first; x[p++]=q->second;}
+	rp[i+1]=p;
+	}
+}
+
+template <typename T>
+CSRMat<T>::CSRMat(const NRMat<T> &rhs)
+: nn(rhs.nrows()), mm(rhs.ncols()), symmetric(false), rowptr(nn+1)
+{
+const typename LA_traits<T>::normtype eps=LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon();
+int nz=0;
+for(SPMatindex i=0; i<nn; ++i) {const T *a=rhs[i]; for(SPMatindex j=0; j<mm; ++j) if(std::abs(a[j])>eps) ++nz;}
+col.resize(nz); v.resize(nz);
+SPMatindex *rp=rowptr, *c=s_wptr(col);
+T *x=s_wptr(v);
+int p=0;
+rp[0]=0;
+for(SPMatindex i=0; i<nn; ++i)
+	{
+	const T *a=rhs[i];
+	for(SPMatindex j=0; j<mm; ++j) if(std::abs(a[j])>eps) {c[p]=j; x[p++]=a[j];}
+	rp[i+1]=p;
+	}
+}
+
+//hermitian as NRSMat: the upper triangle is the conjugate of the stored lower one
+template <typename T>
+CSRMat<T>::CSRMat(const NRSMat<T> &rhs)
+: nn(rhs.nrows()), mm(rhs.ncols()), symmetric(true), rowptr(nn+1)
+{
+const typename LA_traits<T>::normtype eps=LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon();
+int nz=0;
+for(SPMatindex i=0; i<nn; ++i) for(SPMatindex j=0; j<nn; ++j) if(std::abs(rhs(i,j))>eps) ++nz;
+col.resize(nz); v.resize(nz);
+SPMatindex *rp=rowptr, *c=s_wptr(col);
+T *x=s_wptr(v);
+int p=0;
+rp[0]=0;
+for(SPMatindex i=0; i<nn; ++i)
+	{
+	for(SPMatindex j=0; j<nn; ++j)
+		{
+		const T a=rhs(i,j);
+		if(std::abs(a)>eps) {c[p]=j; x[p++]= j>i? LA_traits<T>::conjugate(a) : a;}
+		}
+	rp[i+1]=p;
+	}
+}
+
+template <typename T>
+const T CSRMat<T>::operator()(const SPMatindex i, const SPMatindex j) const
+{
+#ifdef DEBUG
+if(i>=nn || j>=mm) laerror("index out of range in CSRMat::operator()");
+#endif
+if(v.size()==0) return (T)0;
+const SPMatindex *rp=rowptr, *c=col;
+const T *x=v;
+const SPMatindex *q=std::lower_bound(c+rp[i],c+rp[i+1],j);
+if(q!=c+rp[i+1] && *q==j) return x[q-c];
+return (T)0;
+}
+
+template <typename T>
+void CSRMat<T>::put(int fd, bool dimensions, bool transp, bool orcaformat) const
+{
+if(orcaformat) laerror("ORCA format not supported for CSRMat");
+if(transp) {transpose().put(fd,dimensions,false); return;}
+errno=0;
+if(dimensions)
+	{
+	if(sizeof(SPMatindex)!=write(fd,&nn,sizeof(SPMatindex))) laerror("cannot write() 1 in CSRMat::put");
+	if(sizeof(SPMatindex)!=write(fd,&mm,sizeof(SPMatindex))) laerror("cannot write() 2 in CSRMat::put");
+	}
+int sym=symmetric;
+if(sizeof(int)!=write(fd,&sym,sizeof(int))) laerror("cannot write() 3 in CSRMat::put");
+rowptr.put(fd,true);
+col.put(fd,true);
+v.put(fd,true);
+}
+
+template <typename T>
+void CSRMat<T>::get(int fd, bool dimensions, bool transp, bool orcaformat)
+{
+if(orcaformat) laerror("ORCA format not supported for CSRMat");
+errno=0;
+SPMatindex dim[2];
+if(dimensions) {if(2*sizeof(SPMatindex)!=read(fd,&dim,2*sizeof(SPMatindex))) laerror("read() error 1 in CSRMat::get");}
+else {dim[0]= transp?mm:nn; dim[1]= transp?nn:mm;}
+int sym;
+if(sizeof(int)!=read(fd,&sym,sizeof(int))) laerror("read() error 2 in CSRMat::get");
+rowptr.get(fd,true);
+col.get(fd,true);
+v.get(fd,true);
+nn=dim[0]; mm=dim[1]; symmetric=sym;
+if(rowptr.size()!=(nn||mm ? (int)nn+1 : 0) || col.size()!=v.size()) laerror("inconsistent data in CSRMat::get");
+if(transp) transposeme();
+}
+
+template <typename T>
+CSRMat<T> & CSRMat<T>::operator*=(const T &a)
+{
+if(a==(T)0) {clear(); return *this;}
+if(LA_traits<T>::imagpart(a)!=0) symmetric=false;
+if(v.size()) v*=a;
+return *this;
+}
+
+template <typename T>
+CSRMat<T> & CSRMat<T>::operator=(const T &a)
+{
+if(nn!=mm) laerror("assignment of scalar to non-square CSRMat");
+resize(nn,mm);
+symmetric= LA_traits<T>::imagpart(a)==0;
+if(nn==0 || std::abs(a)<=LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) return *this;
+col.resize(nn); v.resize(nn);
+SPMatindex *rp=rowptr, *c=col;
+T *x=v;
+for(SPMatindex i=0; i<nn; ++i) {rp[i]=i; c[i]=i; x[i]=a;}
+rp[nn]=nn;
+return *this;
+}
+
+//in place if all diagonal elements are stored (only the values are copied if shared), otherwise via axpy
+template <typename T>
+CSRMat<T> & CSRMat<T>::operator+=(const T &a)
+{
+if(nn!=mm) laerror("addition of scalar to non-square CSRMat");
+if(a==(T)0) return *this;
+const SPMatindex *rp=s_ptr(rowptr), *c=s_ptr(col);
+bool all= v.size()>0;
+if(all) for(SPMatindex i=0; i<nn; ++i)
+	{
+	const SPMatindex *q=std::lower_bound(c+rp[i],c+rp[i+1],i);
+	if(q==c+rp[i+1] || *q!=i) {all=false; break;}
+	}
+if(!all)
+	{
+	CSRMat<T> d(nn);
+	d=a;
+	const bool sym=symmetric;
+	axpy((T)1,d);
+	symmetric= sym && LA_traits<T>::imagpart(a)==0;
+	return *this;
+	}
+if(LA_traits<T>::imagpart(a)!=0) symmetric=false;
+v.copyonwrite();
+T *x=v;
+for(SPMatindex i=0; i<nn; ++i) x[std::lower_bound(c+rp[i],c+rp[i+1],i)-c]+=a;
+return *this;
+}
+
+//merge of the sorted rows; for the same structure (shared or equal arrays) only the values by BLAS axpy
+//the result is declared hermitian if both are (a matrix without elements counts as hermitian) and alpha is real
+template <typename T>
+void CSRMat<T>::axpy(const T alpha, const CSRMat<T> &x0, const bool transp)
+{
+if((!transp && (nn!=x0.nn || mm!=x0.mm)) || (transp && (nn!=x0.mm || mm!=x0.nn))) laerror("incompatible dimensions for axpy");
+if(alpha==(T)0 || x0.v.size()==0) return;
+CSRMat<T> xt;
+if(transp) xt=x0.transpose();
+const CSRMat<T> &x= transp? xt : x0;
+const bool sym= (v.size()==0? nn==mm : symmetric) && x.symmetric && LA_traits<T>::imagpart(alpha)==0;
+if(v.size()==0)
+	{
+	rowptr=x.rowptr; col=x.col; v=x.v;
+	if(alpha!=(T)1) v*=alpha;
+	symmetric=sym;
+	return;
+	}
+const SPMatindex *rp=rowptr, *c=col, *xrp=x.rowptr, *xc=x.col;
+const int nz=v.size(), xnz=x.v.size();
+if(nz==xnz && ((rp==xrp && c==xc) || (!memcmp(rp,xrp,(nn+1)*sizeof(SPMatindex)) && !memcmp(c,xc,nz*sizeof(SPMatindex)))))
+	{
+	v.axpy(alpha,x.v);
+	symmetric=sym;
+	return;
+	}
+const T *a=v, *b=x.v;
+NRVec<SPMatindex> nrp(nn+1);
+SPMatindex *np=nrp;
+np[0]=0;
+for(SPMatindex i=0; i<nn; ++i)
+	{
+	SPMatindex p=rp[i], q=xrp[i], cnt=0;
+	while(p<rp[i+1] && q<xrp[i+1]) {if(c[p]<xc[q]) ++p; else if(c[p]>xc[q]) ++q; else {++p; ++q;} ++cnt;}
+	np[i+1]=np[i]+cnt+(rp[i+1]-p)+(xrp[i+1]-q);
+	}
+NRVec<SPMatindex> ncol(np[nn]);
+NRVec<T> nv(np[nn]);
+SPMatindex *nc=ncol;
+T *y=nv;
+int k=0;
+for(SPMatindex i=0; i<nn; ++i)
+	{
+	SPMatindex p=rp[i], q=xrp[i];
+	while(p<rp[i+1] || q<xrp[i+1])
+		{
+		if(q==xrp[i+1] || (p<rp[i+1] && c[p]<xc[q])) {nc[k]=c[p]; y[k++]=a[p++];}
+		else if(p==rp[i+1] || c[p]>xc[q]) {nc[k]=xc[q]; y[k++]=alpha*b[q++];}
+		else {nc[k]=c[p]; y[k++]=a[p++]+alpha*b[q++];}
+		}
+	}
+rowptr=nrp; col=ncol; v=nv;
+symmetric=sym;
+}
+
+template <typename T>
+const typename LA_traits<T>::normtype CSRMat<T>::norm(const T scalar) const
+{
+if(scalar==(T)0) return v.size()? v.norm() : (typename LA_traits<T>::normtype)0;
+if(nn!=mm) laerror("subtraction of scalar from non-square CSRMat in norm()");
+const SPMatindex *rp=s_ptr(rowptr), *c=s_ptr(col);
+const T *x=s_ptr(v);
+typename LA_traits<T>::normtype sum=0;
+for(SPMatindex i=0; i<nn; ++i)
+	{
+	bool d=false;
+	if(x) for(SPMatindex k=rp[i]; k<rp[i+1]; ++k)
+		{
+		T e=x[k];
+		if(c[k]==i) {e-=scalar; d=true;}
+		sum+=LA_traits<T>::sqrabs(e);
+		}
+	if(!d) sum+=LA_traits<T>::sqrabs(scalar);
+	}
+return std::sqrt(sum);
+}
+
+template <typename T>
+const T CSRMat<T>::dot(const CSRMat<T> &rhs) const
+{
+if(nn!=rhs.nn || mm!=rhs.mm) laerror("dot of incompatible CSRMat matrices");
+if(v.size()==0 || rhs.v.size()==0) return (T)0;
+const SPMatindex *rp=rowptr, *c=col, *xrp=rhs.rowptr, *xc=rhs.col;
+if(rp==xrp && c==xc) return v.dot(rhs.v);
+const T *a=v, *b=rhs.v;
+T s=0;
+for(SPMatindex i=0; i<nn; ++i)
+	{
+	SPMatindex p=rp[i], q=xrp[i];
+	while(p<rp[i+1] && q<xrp[i+1])
+		{
+		if(c[p]<xc[q]) ++p;
+		else if(c[p]>xc[q]) ++q;
+		else s+=LA_traits<T>::conjugate(a[p++])*b[q++];
+		}
+	}
+return s;
+}
+
+template <typename T>
+const T CSRMat<T>::dot(const NRMat<T> &rhs) const
+{
+if((int)nn!=rhs.nrows() || (int)mm!=rhs.ncols()) laerror("dot of incompatible matrices");
+if(v.size()==0) return (T)0;
+const SPMatindex *rp=rowptr, *c=col;
+const T *a=v;
+T s=0;
+for(SPMatindex i=0; i<nn; ++i)
+	{
+	const T *b=rhs[i];
+	for(SPMatindex k=rp[i]; k<rp[i+1]; ++k) s+=LA_traits<T>::conjugate(a[k])*b[c[k]];
+	}
+return s;
+}
+
+template <typename T>
+const T CSRMat<T>::trace() const
+{
+if(v.size()==0) return (T)0;
+const SPMatindex *rp=rowptr, *c=col;
+const T *x=v;
+T s=0;
+for(SPMatindex i=0; i<nn && i<mm; ++i)
+	{
+	const SPMatindex *q=std::lower_bound(c+rp[i],c+rp[i+1],i);
+	if(q!=c+rp[i+1] && *q==i) s+=x[q-c];
+	}
+return s;
+}
+
+//sparse times sparse, row-wise with a dense accumulator: F. G. Gustavson, ACM Trans. Math. Softw. 4, 250 (1978)
+//two passes (count, then compute); the columns of each result row are sorted; the result is not declared hermitian
+template <typename T>
+void CSRMat<T>::gemm(const T beta, const CSRMat<T> &a0, const char transa, const CSRMat<T> &b0, const char transb, const T alpha)
+{
+const char ta=tolower(transa), tb=tolower(transb);
+CSRMat<T> at, bt;
+if(ta!='n') at=a0.transpose(ta=='c');
+if(tb!='n') bt=b0.transpose(tb=='c');
+const CSRMat<T> &a= ta!='n'? at : a0;
+const CSRMat<T> &b= tb!='n'? bt : b0;
+if(a.mm!=b.nn) laerror("incompatible matrices in CSRMat::gemm");
+if(beta!=(T)0 && (nn!=a.nn || mm!=b.mm)) laerror("incompatible result matrix in CSRMat::gemm");
+const SPMatindex n=a.nn, m=b.mm;
+CSRMat<T> r(n,m);
+if(alpha!=(T)0 && a.v.size() && b.v.size())
+	{
+	const SPMatindex *arp=a.rowptr, *ac=a.col, *brp=b.rowptr, *bc=b.col;
+	const T *av=a.v, *bv=b.v;
+	NRVec<SPMatindex> marker(m);
+	SPMatindex *mk=marker;
+	for(SPMatindex j=0; j<m; ++j) mk[j]=n;
+	SPMatindex *rp=r.rowptr;
+	for(SPMatindex i=0; i<n; ++i)
+		{
+		SPMatindex cnt=0;
+		for(SPMatindex k=arp[i]; k<arp[i+1]; ++k) for(SPMatindex q=brp[ac[k]]; q<brp[ac[k]+1]; ++q) if(mk[bc[q]]!=i) {mk[bc[q]]=i; ++cnt;}
+		rp[i+1]=rp[i]+cnt;
+		}
+	r.col.resize(rp[n]); r.v.resize(rp[n]);
+	if(rp[n])
+		{
+		SPMatindex *c=r.col;
+		T *y=r.v;
+		NRVec<T> acc(m);
+		T *ac2=acc;
+		for(SPMatindex j=0; j<m; ++j) mk[j]=n;
+		for(SPMatindex i=0; i<n; ++i)
+			{
+			SPMatindex len=0;
+			for(SPMatindex k=arp[i]; k<arp[i+1]; ++k)
+				{
+				const T x=av[k];
+				for(SPMatindex q=brp[ac[k]]; q<brp[ac[k]+1]; ++q)
+					{
+					const SPMatindex j=bc[q];
+					if(mk[j]!=i) {mk[j]=i; c[rp[i]+len++]=j; ac2[j]=x*bv[q];}
+					else ac2[j]+=x*bv[q];
+					}
+				}
+			std::sort(c+rp[i],c+rp[i]+len);
+			for(SPMatindex t=rp[i]; t<rp[i]+len; ++t) y[t]=alpha*ac2[c[t]];
+			}
+		}
+	}
+if(beta==(T)0) *this=r;
+else
+	{
+	*this*=beta;
+	axpy((T)1,r);
+	}
+symmetric=false;
+}
+
+template <typename T>
+CSRMat<T> & CSRMat<T>::oplusequal(const CSRMat<T> &rhs)
+{
+const bool sym= (v.size()==0? nn==mm : symmetric) && rhs.symmetric;
+const SPMatindex n=nn+rhs.nn, m=mm+rhs.mm;
+const int nz=v.size(), rnz=rhs.v.size();
+NRVec<SPMatindex> nrp(n+1), ncol(nz+rnz);
+NRVec<T> nv(nz+rnz);
+SPMatindex *np=nrp, *nc=s_wptr(ncol);
+T *y=s_wptr(nv);
+const SPMatindex *rp=s_ptr(rowptr), *c=s_ptr(col), *rrp=s_ptr(rhs.rowptr), *rc=s_ptr(rhs.col);
+const T *x=s_ptr(v), *rx=s_ptr(rhs.v);
+for(SPMatindex i=0; i<=nn; ++i) np[i]= rp? rp[i] : 0;
+for(SPMatindex i=1; i<=rhs.nn; ++i) np[nn+i]=nz+rrp[i];
+for(int k=0; k<nz; ++k) {nc[k]=c[k]; y[k]=x[k];}
+for(int k=0; k<rnz; ++k) {nc[nz+k]=mm+rc[k]; y[nz+k]=rx[k];}
+nn=n; mm=m;
+rowptr=nrp; col=ncol; v=nv;
+symmetric=sym;
+return *this;
+}
+
+template <typename T>
+const CSRMat<T> CSRMat<T>::otimes(const CSRMat<T> &rhs) const
+{
+CSRMat<T> r(nn*rhs.nn,mm*rhs.mm);
+r.symmetric= symmetric && rhs.symmetric;
+const int nz=v.size(), rnz=rhs.v.size();
+if(nz==0 || rnz==0) return r;
+const SPMatindex *rp=rowptr, *c=col, *brp=rhs.rowptr, *bc=rhs.col;
+const T *x=v, *bx=rhs.v;
+r.col.resize(nz*rnz); r.v.resize(nz*rnz);
+SPMatindex *np=r.rowptr, *nc=r.col;
+T *y=r.v;
+int p=0;
+for(SPMatindex i=0; i<nn; ++i) for(SPMatindex k=0; k<rhs.nn; ++k)
+	{
+	for(SPMatindex a=rp[i]; a<rp[i+1]; ++a) for(SPMatindex b=brp[k]; b<brp[k+1]; ++b) {nc[p]=c[a]*rhs.mm+bc[b]; y[p++]=x[a]*bx[b];}
+	np[i*rhs.nn+k+1]=p;
+	}
+return r;
+}
+
+//row r becomes row to[r]: the rows are moved as blocks, their columns stay sorted
+template <typename T>
+void CSRMat<T>::permrows(const SPMatindex *to)
+{
+const int nz=v.size();
+NRVec<SPMatindex> nrp(nn+1);
+SPMatindex *np=nrp;
+const SPMatindex *rp=rowptr;
+np[0]=0;
+for(SPMatindex r=0; r<nn; ++r) np[to[r]+1]=rp[r+1]-rp[r];
+for(SPMatindex r=0; r<nn; ++r) np[r+1]+=np[r];
+if(nz)
+	{
+	NRVec<SPMatindex> ncol(nz);
+	NRVec<T> nv(nz);
+	SPMatindex *nc=ncol;
+	T *y=nv;
+	const SPMatindex *c=col;
+	const T *x=v;
+	for(SPMatindex r=0; r<nn; ++r) for(SPMatindex k=rp[r], t=np[to[r]]; k<rp[r+1]; ++k, ++t) {nc[t]=c[k]; y[t]=x[k];}
+	col=ncol; v=nv;
+	}
+rowptr=nrp;
+}
+
+template <typename T>
+void CSRMat<T>::permuteme_rows(const NRPerm<int> &p, const bool inverse)
+{
+if((SPMatindex)p.size()!=nn) laerror("inconsistent dimension in permuteme_rows");
+NRPerm<int> pinv;
+if(inverse) pinv=p.inverse();
+const NRPerm<int> &pp= inverse?pinv:p;
+NRVec<SPMatindex> to(nn);
+for(SPMatindex r=0; r<nn; ++r) to[r]=pp[r+1]-1;
+if(nn) permrows(to);
+symmetric=false;
+}
+
+//columns via the transposed matrix (counting sorts keep the rows sorted)
+template <typename T>
+void CSRMat<T>::permuteme_cols(const NRPerm<int> &p, const bool inverse)
+{
+if((SPMatindex)p.size()!=mm) laerror("inconsistent dimension in permuteme_cols");
+*this=transpose();
+permuteme_rows(p,inverse);
+*this=transpose();
+}
+
+template <typename T>
+void CSRMat<T>::permuteme_both(const NRPerm<int> &p, const NRPerm<int> &q, const bool inverse)
+{
+const bool sym= symmetric && p==q;
+permuteme_rows(p,inverse);
+permuteme_cols(q,inverse);
+symmetric=sym;
+}
+
+template <typename T>
+void CSRMat<T>::permuterows(const NRVec<SPMatindex> &p)
+{
+if((SPMatindex)p.size()!=nn) laerror("inconsistent dimension in permuterows");
+if(nn) permrows(p);
+symmetric=false;
+}
+
+template <typename T>
+void CSRMat<T>::permutecolumns(const NRVec<SPMatindex> &p)
+{
+if((SPMatindex)p.size()!=mm) laerror("inconsistent dimension in permutecolumns");
+*this=transpose();
+permuterows(p);
+*this=transpose();
+}
+
+template <typename T>
+void CSRMat<T>::permuteindices(const NRVec<SPMatindex> &p)
+{
+if((SPMatindex)p.size()!=nn || (SPMatindex)p.size()!=mm) laerror("inconsistent dimension in permuteindices");
+const bool sym=symmetric;
+permuterows(p);
+permutecolumns(p);
+symmetric=sym;
+}
+
+//square: diagonal (binary search in each row); non-square: diagonal of A^T A (for preconditioning as SparseMat::diagonalof); cache is ignored
+template <typename T>
+const T* CSRMat<T>::diagonalof(NRVec<T> &r, const bool divide, bool cache) const
+{
+#ifdef DEBUG
+if((int)mm!=r.size()) laerror("incompatible vector size in diagonalof()");
+#endif
+NOT_GPU(r);
+r.copyonwrite();
+T *rr=r;
+const SPMatindex *rp=s_ptr(rowptr), *c=s_ptr(col);
+const T *x=s_ptr(v);
+if(nn==mm)
+	{
+	for(SPMatindex i=0; i<nn; ++i)
+		{
+		T d=0;
+		if(x)
+			{
+			const SPMatindex *q=std::lower_bound(c+rp[i],c+rp[i+1],i);
+			if(q!=c+rp[i+1] && *q==i) d=x[q-c];
+			}
+		if(divide) {if(d!=(T)0) rr[i]/=d;}
+		else rr[i]=d;
+		}
+	}
+else
+	{
+	NRVec<T> d(mm);
+	d.clear();
+	T *dd=d;
+	for(int k=0; k<v.size(); ++k) dd[c[k]]+=x[k]*x[k];
+	if(divide) {for(SPMatindex j=0; j<mm; ++j) if(dd[j]!=(T)0) rr[j]/=dd[j];}
+	else for(SPMatindex j=0; j<mm; ++j) rr[j]=dd[j];
+	}
+return divide?NULL:rr;
+}
+
+//r = beta r + alpha op(A) x, op = 'n', 't' or 'c' (conjugate transpose, the same as 't' for real types)
+template <typename T>
+void NRVec<T>::gemv(const T beta, const CSRMat<T> &a, const char trans, const T alpha, const NRVec<T> &x)
+{
+const char tr=tolower(trans);
+const bool tp= tr!='n';
+const bool cj= tr=='c' && LA_traits<T>::is_complex();
+if((tp?a.nrows():a.ncols())!=(SPMatindex)x.size()) laerror("incompatible sizes in gemv");
+#ifdef DEBUG
+if((tp?a.ncols():a.nrows())!=(SPMatindex)nn) laerror("incompatible result vector in gemv");
+#endif
+NOT_GPU(*this);
+NOT_GPU(x);
+copyonwrite();
+if(beta==(T)0) clear();
+else if(beta!=(T)1) (*this)*=beta;
+if(alpha==(T)0 || a.length()==0) return;
+const SPMatindex *rp=a.getrowptr(), *c=a.getcol();
+const T *av=a.getval();
+const T *xv=x.v;
+const SPMatindex n=a.nrows();
+if(!tp)
+	for(SPMatindex i=0; i<n; ++i)
+		{
+		T s=0;
+		for(SPMatindex k=rp[i]; k<rp[i+1]; ++k) s+=av[k]*xv[c[k]];
+		v[i]+=alpha*s;
+		}
+else
+	for(SPMatindex i=0; i<n; ++i)
+		{
+		const T ax=alpha*xv[i];
+		if(cj) for(SPMatindex k=rp[i]; k<rp[i+1]; ++k) v[c[k]]+=LA_traits<T>::conjugate(av[k])*ax;
+		else for(SPMatindex k=rp[i]; k<rp[i+1]; ++k) v[c[k]]+=av[k]*ax;
+		}
+}
+
+//vector from a matrix with one row or one column
+template <typename T>
+NRVec<T>::NRVec(const CSRMat<T> &rhs)
+: NRVec(rhs.nrows()>1?rhs.nrows():rhs.ncols())
+{
+if(rhs.nrows()>1 && rhs.ncols()>1) laerror("cannot construct a vector from a sparse matrix with more than one row/column");
+clear();
+const SPMatindex *rp=s_ptr(rhs.getrowptr()), *c=s_ptr(rhs.getcol());
+const T *x=s_ptr(rhs.getval());
+if(!x) return;
+if(rhs.nrows()>1) {for(SPMatindex i=0; i<rhs.nrows(); ++i) if(rp[i+1]>rp[i]) v[i]=x[rp[i]];}
+else for(SPMatindex k=0; k<rp[1]; ++k) v[c[k]]=x[k];
+}
+
+//conversions to the other classes
+
+template <typename T>
+NRMat<T>::NRMat(const CSRMat<T> &rhs)
+{
+nn=rhs.nrows();
+mm=rhs.ncols();
+count=new int(1);
+#ifdef CUDALA
+location=cpu;
+#endif
+T *p;
+#ifdef MATPTR
+	v= new T*[nn];
+	p=v[0] = new T[mm*nn];
+	for (int i=1; i< nn; i++) v[i] = v[i-1] + mm;
+#else
+	p= v = new T[mm*nn];
+#endif
+memset(p,0,nn*mm*sizeof(T));
+const SPMatindex *rp=s_ptr(rhs.getrowptr()), *c=s_ptr(rhs.getcol());
+const T *x=s_ptr(rhs.getval());
+if(x) for(int i=0; i<nn; ++i) for(SPMatindex k=rp[i]; k<rp[i+1]; ++k) p[(size_t)i*mm+c[k]]=x[k];
+}
+
+template <typename T>
+NRMat<T> & NRMat<T>::operator+=(const CSRMat<T> &rhs)
+{
+if(nn!=(int)rhs.nrows() || mm!=(int)rhs.ncols()) laerror("incompatible matrices in +=");
+NOT_GPU(*this);
+copyonwrite();
+const SPMatindex *rp=s_ptr(rhs.getrowptr()), *c=s_ptr(rhs.getcol());
+const T *x=s_ptr(rhs.getval());
+if(x) for(int i=0; i<nn; ++i) {T *a=(*this)[i]; for(SPMatindex k=rp[i]; k<rp[i+1]; ++k) a[c[k]]+=x[k];}
+return *this;
+}
+
+template <typename T>
+NRMat<T> & NRMat<T>::operator-=(const CSRMat<T> &rhs)
+{
+if(nn!=(int)rhs.nrows() || mm!=(int)rhs.ncols()) laerror("incompatible matrices in -=");
+NOT_GPU(*this);
+copyonwrite();
+const SPMatindex *rp=s_ptr(rhs.getrowptr()), *c=s_ptr(rhs.getcol());
+const T *x=s_ptr(rhs.getval());
+if(x) for(int i=0; i<nn; ++i) {T *a=(*this)[i]; for(SPMatindex k=rp[i]; k<rp[i+1]; ++k) a[c[k]]-=x[k];}
+return *this;
+}
+
+//C = beta C + alpha op(A) op(B), A sparse: rows of op(B) accumulated (op(B) transposed first if needed)
+template <typename T>
+void NRMat<T>::gemm(const T &beta, const CSRMat<T> &a, const char transa, const NRMat<T> &b, const char transb, const T &alpha)
+{
+const char ta=tolower(transa), tb=tolower(transb);
+const bool tpa= ta!='n', cja= ta=='c' && LA_traits<T>::is_complex();
+const int an= tpa? a.ncols() : a.nrows(), ak= tpa? a.nrows() : a.ncols();
+const int bk= tb!='n'? b.ncols() : b.nrows(), bm= tb!='n'? b.nrows() : b.ncols();
+if(ak!=bk) laerror("incompatible matrices in NRMat::gemm with CSRMat");
+#ifdef DEBUG
+if(nn!=an || mm!=bm) laerror("incompatible result matrix in NRMat::gemm with CSRMat");
+#endif
+NOT_GPU(*this);
+copyonwrite();
+if(beta==(T)0) clear();
+else if(beta!=(T)1) (*this)*=beta;
+if(alpha==(T)0 || a.length()==0) return;
+NRMat<T> bt;
+if(tb!='n') bt=b.transpose(tb=='c');
+const NRMat<T> &bb= tb!='n'? bt : b;
+const SPMatindex *rp=a.getrowptr(), *c=a.getcol();
+const T *x=a.getval();
+for(SPMatindex i=0; i<a.nrows(); ++i) for(SPMatindex k=rp[i]; k<rp[i+1]; ++k)
+	{
+	const T s= alpha*(cja? LA_traits<T>::conjugate(x[k]) : x[k]);
+	T *d= (*this)[tpa? c[k] : i];
+	const T *e= bb[tpa? i : c[k]];
+	for(int j=0; j<mm; ++j) d[j]+=s*e[j];
+	}
+}
+
+template <typename T>
+const NRMat<T> NRMat<T>::operator*(const CSRMat<T> &rhs) const
+{
+if((SPMatindex)mm!=rhs.nrows()) laerror("incompatible matrices in NRMat*CSRMat");
+NOT_GPU(*this);
+NRMat<T> r(nn,rhs.ncols());
+r.clear();
+if(rhs.length()==0) return r;
+const SPMatindex *rp=rhs.getrowptr(), *c=rhs.getcol();
+const T *x=rhs.getval();
+for(int i=0; i<nn; ++i)
+	{
+	const T *a=(*this)[i];
+	T *d=r[i];
+	for(int k=0; k<mm; ++k) if(a[k]!=(T)0) for(SPMatindex q=rp[k]; q<rp[k+1]; ++q) d[c[q]]+=a[k]*x[q];
+	}
+return r;
+}
+
+//lower triangle, as from SparseSMat
+template <typename T>
+NRSMat<T>::NRSMat(const CSRMat<T> &rhs)
+: nn(rhs.nrows())
+{
+if(rhs.nrows()!=rhs.ncols()) laerror("cannot transform rectangular matrix to NRSMat");
+#ifdef CUDALA
+location=cpu;
+#endif
+count=new int(1);
+v=new T[NN2];
+memset(v,0,NN2*sizeof(T));
+const SPMatindex *rp=s_ptr(rhs.getrowptr()), *c=s_ptr(rhs.getcol());
+const T *x=s_ptr(rhs.getval());
+if(x) for(int i=0; i<nn; ++i) for(SPMatindex k=rp[i]; k<rp[i+1] && (int)c[k]<=i; ++k) v[(size_t)i*(i+1)/2+c[k]]=x[k];
+}
+
+template <typename T>
+SparseMat<T>::SparseMat(const CSRMat<T> &rhs)
+: nn(rhs.nrows()), mm(rhs.ncols()), symmetric(0), nonzero(0), count(new int(1)), list(NULL), rowsorted(NULL), colsorted(NULL)
+{
+for(typename CSRMat<T>::iterator p=rhs.begin(); p.notend(); ++p) add(p->row,p->col,p->elem);
+}
+
+template <typename T>
+SparseSMat<T>::SparseSMat(const CSRMat<T> &rhs)
+: nn(rhs.nrows()), mm(rhs.ncols()), count(new int(1))
+{
+v= new std::map<SPMatindex,T> * [nn];
+memset(v,0,nn*sizeof(std::map<SPMatindex,T> *));
+for(typename CSRMat<T>::iterator p=rhs.begin(); p.notend(); ++p) add(p->row,p->col,p->elem,false);
+}
 
 
+//real and imaginary parts, complex matrices from real ones (nonclass.h): the index arrays stay shared,
+//unless elements which became zero have to be removed
+template<typename R>
+static const CSRMat<R> s_part(const CSRMat<std::complex<R> > &a, const NRVec<R> &x, const bool sym)
+{
+CSRMat<R> r(a.nrows(),a.ncols());
+if(a.length())
+	{
+	r=CSRMat<R>(a.nrows(),a.ncols(),a.getrowptr(),a.getcol(),x);
+	const R eps=LA_precision<R>::sparse_epsilon();
+	const R *y=x;
+	for(int k=0; k<x.size(); ++k) if(std::abs(y[k])<=eps) {r.simplify(); break;}
+	}
+if(sym && a.nrows()==a.ncols()) r.setsymmetric();
+return r;
+}
+
+template<typename R>
+static const CSRMat<std::complex<R> > s_cmplx(const CSRMat<R> &a, const bool imag)
+{
+CSRMat<std::complex<R> > r(a.nrows(),a.ncols());
+if(a.length()) r=CSRMat<std::complex<R> >(a.nrows(),a.ncols(),a.getrowptr(),a.getcol(),NRVec<std::complex<R> >(a.getval(),imag));
+if(!imag && a.issymmetric()) r.setsymmetric();
+return r;
+}
+
+template<>
+const CSRMat<double> realpart<CSRMat<std::complex<double> > >(const CSRMat<std::complex<double> > &a)
+{return s_part(a,a.length()? realpart(a.getval()) : NRVec<double>(),a.issymmetric());}
+
+template<>
+const CSRMat<float> realpart<CSRMat<std::complex<float> > >(const CSRMat<std::complex<float> > &a)
+{return s_part(a,a.length()? realpart(a.getval()) : NRVec<float>(),a.issymmetric());}
+
+template<>
+const CSRMat<double> imagpart<CSRMat<std::complex<double> > >(const CSRMat<std::complex<double> > &a)
+{return s_part(a,a.length()? imagpart(a.getval()) : NRVec<double>(),false);}
+
+template<>
+const CSRMat<float> imagpart<CSRMat<std::complex<float> > >(const CSRMat<std::complex<float> > &a)
+{return s_part(a,a.length()? imagpart(a.getval()) : NRVec<float>(),false);}
+
+template<>
+const CSRMat<std::complex<double> > realmatrix<CSRMat<double> >(const CSRMat<double> &a)
+{return s_cmplx(a,false);}
+
+template<>
+const CSRMat<std::complex<float> > realmatrix<CSRMat<float> >(const CSRMat<float> &a)
+{return s_cmplx(a,false);}
+
+template<>
+const CSRMat<std::complex<double> > imagmatrix<CSRMat<double> >(const CSRMat<double> &a)
+{return s_cmplx(a,true);}
+
+template<>
+const CSRMat<std::complex<float> > imagmatrix<CSRMat<float> >(const CSRMat<float> &a)
+{return s_cmplx(a,true);}
+
+//re + i im: a BLAS axpy of the values if both have the same structure, otherwise a merge
+template<>
+const CSRMat<std::complex<double> > complexmatrix<CSRMat<double> >(const CSRMat<double> &re, const CSRMat<double> &im)
+{CSRMat<std::complex<double> > r=s_cmplx(re,false); r.axpy(std::complex<double>(0,1),s_cmplx(im,false)); return r;}
+
+template<>
+const CSRMat<std::complex<float> > complexmatrix<CSRMat<float> >(const CSRMat<float> &re, const CSRMat<float> &im)
+{CSRMat<std::complex<float> > r=s_cmplx(re,false); r.axpy(std::complex<float>(0,1),s_cmplx(im,false)); return r;}
 
-/*
- Commented out by Roman for ICC
 
 #define INSTANTIZE(T) \
-template void CSRMat<T>::gemm(const T beta, const CSRMat &a, const char transa, const CSRMat &b, const char transb, const T alpha); \
+template class CSRMat<T>; \
-template CSRMat<T> & CSRMat<T>::operator*=(const T &a); \
+template NRMat<T>::NRMat(const CSRMat<T> &rhs); \
-template void CSRMat<T>::gemv(const T beta, NRVec<T> &r, const char trans, const T alpha, const NRVec<T> &x) const; \
+template NRSMat<T>::NRSMat(const CSRMat<T> &rhs); \
-template void CSRMat<T>::axpy(const T alpha, const CSRMat &x, const bool transp); \
+template SparseMat<T>::SparseMat(const CSRMat<T> &rhs); \
-template CSRMat<T> & CSRMat<T>::operator=(const T &a); \
+template SparseSMat<T>::SparseSMat(const CSRMat<T> &rhs); \
-template CSRMat<T> & CSRMat<T>::operator+=(const T &a); \
+template void NRVec<T>::gemv(const T beta, const CSRMat<T> &a, const char trans, const T alpha, const NRVec<T> &x); \
-template CSRMat<T> & CSRMat<T>::operator-=(const T &a); \
+template NRVec<T>::NRVec(const CSRMat<T> &rhs); \
-template LA_traits<T>::normtype CSRMat<T>::norm(const T scalar) const; \
+template NRMat<T> & NRMat<T>::operator+=(const CSRMat<T> &rhs); \
-template const T* CSRMat<T>::diagonalof(NRVec<T> &r, const bool divide, bool cache) const; \
+template NRMat<T> & NRMat<T>::operator-=(const CSRMat<T> &rhs); \
-template void CSRMat<T>::get(int fd, bool dimen, bool transp); \
+template void NRMat<T>::gemm(const T &beta, const CSRMat<T> &a, const char transa, const NRMat<T> &b, const char transb, const T &alpha); \
-template void CSRMat<T>::put(int fd, bool dimen, bool transp) const; \
+template const NRMat<T> NRMat<T>::operator*(const CSRMat<T> &rhs) const; \
-
 
 INSTANTIZE(double)
+INSTANTIZE(float)
 INSTANTIZE(std::complex<double>)
-*/
+INSTANTIZE(std::complex<float>)
-
-//// forced instantization of functions in the header in the corresponding object file
-//@@@@@template class CSRMat<double>;
-//@@@@template class CSRMat<complex<double> >;
-//@@@@ unfinished class commented out
-
-
 
 }//namespace

csrmat.h

@@ -1,6 +1,6 @@
 /*
     LA: linear algebra C++ interface library
-    Copyright (C) 2010 Jiri Pittner <jiri.pittner@jh-inst.cas.cz> or <jiri@pittnerovi.com>
+    Copyright (C) 2010-2026 Jiri Pittner <jiri.pittner@jh-inst.cas.cz> or <jiri@pittnerovi.com>
 
     This program is free software: you can redistribute it and/or modify
     it under the terms of the GNU General Public License as published by
@@ -32,16 +32,13 @@
 #include "vec.h"
 #include "mat.h"
 #include "smat.h"
-#include "qsort.h"
-
-#ifdef CUDALA
-#include "cusparse.h"
-#include "cuda_runtime.h"
-#endif
 
 namespace LA {
 
 //compressed row format read-efficient matrix storage
+//row i has the elements v[k], column col[k], k=rowptr[i]..rowptr[i+1]-1, columns strictly ascending within a row
+//the full matrix is always stored; symmetric only declares A = A^dagger (hermitian, real symmetric) e.g. for davidson, lanczos
+//rowptr, col and v share data independently (copy-on-write), e.g. a scaled copy shares the structure with the original
 
 template <typename T>
 class CSRMat
@@ -49,92 +46,157 @@ class CSRMat
 protected:
 	SPMatindex nn;
 	SPMatindex mm;
-	SPMatindex nnz;
+	bool symmetric;
-	int *count;
+	NRVec<SPMatindex> rowptr;
-	SPMatindex *rowptr;
+	NRVec<SPMatindex> col;
-	SPMatindex *col;
+	NRVec<T> v;
-	T *v;
+	void fromcoo(const SPMatindex n, const SPMatindex m, const int nz, const SPMatindex *ri, const SPMatindex *ci, const T *vi); //sorts, sums duplicates, drops small elements
-#ifdef CUDALA
+	void permrows(const SPMatindex *to); //row r becomes row to[r]
-        GPUID location;
-	cusparseMatDescr_t descr;
-#endif
-
 public:
-	CSRMat() : nn(0), mm(0), nnz(0), count(NULL), rowptr(NULL), col(NULL), v(NULL) {};
+	//iterator over the stored elements in row-major order
-	CSRMat(const CSRMat &rhs);
+	class iterator {
-	explicit CSRMat(const SPMatindex n, const SPMatindex m); 
+	private:
-        explicit CSRMat(const SPMatindex n);
+		const CSRMat *a;
+		SPMatindex row;
+		int k;
+		matel<T> my;
+		void set() {while(row<a->nn && k>=(int)a->rowptr[row+1]) ++row; if(row<a->nn) {my.row=row; my.col=a->col[k]; my.elem=a->v[k]; my.next=NULL;}}
+	public:
+		iterator(): a(NULL), row(0), k(0) {};
+		iterator(const CSRMat &rhs): a(&rhs), row(0), k(0) {if(a->nn) set();}
+		iterator & operator++() {++k; set(); return *this;}
+		bool notend() const {return a && row<a->nn;}
+		bool operator==(const iterator &rhs) const {return notend()==rhs.notend() && (!notend() || k==rhs.k);}
+		bool operator!=(const iterator &rhs) const {return !(*this==rhs);}
+		const matel<T> & operator*() const {return my;}
+		const matel<T> * operator->() const {return &my;}
+	};
+	iterator begin() const {return iterator(*this);}
+	iterator end() const {return iterator();}
+
+	CSRMat() : nn(0), mm(0), symmetric(false) {};
+	explicit CSRMat(const SPMatindex n, const SPMatindex m); //zero matrix
+	explicit CSRMat(const SPMatindex n); //zero square matrix
+	CSRMat(const SPMatindex n, const SPMatindex m, const NRVec<SPMatindex> &rowptr, const NRVec<SPMatindex> &col, const NRVec<T> &v); //shares the arrays
 	explicit CSRMat(const SparseMat<T> &rhs);
 	explicit CSRMat(const SparseSMat<T> &rhs);
 	explicit CSRMat(const NRSMat<T> &rhs);
 	explicit CSRMat(const NRMat<T> &rhs);
-	CSRMat & operator=(const CSRMat &rhs);
+	void copyonwrite(bool detachonly=false, bool deep=true);
-	void copyonwrite();
+	void resize(const SPMatindex n, const SPMatindex m); //zero matrix
-        void resize(const SPMatindex nn, const SPMatindex mm);
 	void dealloc(void) {resize(0,0);}
-	void moveto(GPUID destination);
+	void clear() {bool sym=symmetric; resize(nn,mm); symmetric=sym;} //zero matrix of the same dimensions and symmetry
-        void clear();
+	void swap(CSRMat &rhs)
-	~CSRMat();
+		{
-	inline int getcount() const {return count?*count:0;}
+		SPMatindex tmpnn=nn; nn=rhs.nn; rhs.nn=tmpnn;
+		SPMatindex tmpmm=mm; mm=rhs.mm; rhs.mm=tmpmm;
+		bool tmpsymmetric=symmetric; symmetric=rhs.symmetric; rhs.symmetric=tmpsymmetric;
+		rowptr.swap(rhs.rowptr); col.swap(rhs.col); v.swap(rhs.v);
+		}
+	inline SPMatindex nrows() const {return nn;}
+	inline SPMatindex ncols() const {return mm;}
+	inline int length() const {return v.size();} //number of stored elements
+	inline bool issymmetric() const {return symmetric;}
+	inline void setsymmetric() {if(nn!=mm) laerror("non-square cannot be symmetric"); symmetric=true;} //only declares the matrix hermitian
+	inline void setunsymmetric() {symmetric=false;}
+	inline const NRVec<SPMatindex> & getrowptr() const {return rowptr;}
+	inline const NRVec<SPMatindex> & getcol() const {return col;}
+	inline const NRVec<T> & getval() const {return v;}
+	const T operator()(const SPMatindex i, const SPMatindex j) const; //element (zero if not stored), binary search in the row
+	void gemv(const T beta, NRVec<T> &r, const char trans, const T alpha, const NRVec<T> &x) const {r.gemv(beta,*this,trans,alpha,x);};
+	inline const NRVec<T> operator*(const NRVec<T> &rhs) const {NRVec<T> result(nn); result.gemv((T)0,*this,'n',(T)1,rhs); return result;};
+	const T* diagonalof(NRVec<T> &r, const bool divide=0, bool cache=false) const; //diagonal (diagonal of A^T A for non-square, as SparseMat), or divide r by it
+	const CSRMat transpose(bool conj=false) const;
+	CSRMat & operator*=(const T &a); //multiply by a scalar (the structure stays shared)
+	CSRMat & operator=(const T &a); //a times the unit matrix
+	CSRMat & operator+=(const T &a); //add a to the diagonal
+	CSRMat & operator-=(const T &a) {return *this += -a;}
+	void identity() {*this = (T)1;}
+	void axpy(const T alpha, const CSRMat &x, const bool transp=false); //this += alpha x (alpha x^T); elements cancelled to zero stay stored, see simplify()
+	CSRMat & operator+=(const CSRMat &rhs) {axpy((T)1,rhs); return *this;}
+	CSRMat & operator-=(const CSRMat &rhs) {axpy((T)-1,rhs); return *this;}
 	inline const CSRMat operator*(const T &rhs) const {return CSRMat(*this) *= rhs;}
 	inline const CSRMat operator+(const T &rhs) const {return CSRMat(*this) += rhs;}
 	inline const CSRMat operator-(const T &rhs) const {return CSRMat(*this) -= rhs;}
 	inline const CSRMat operator+(const CSRMat &rhs) const {return CSRMat(*this) += rhs;}
 	inline const CSRMat operator-(const CSRMat &rhs) const {return CSRMat(*this) -= rhs;}
-        CSRMat & operator*=(const T &a);         //multiply by a scalar
+	const typename LA_traits<T>::normtype norm(const T scalar=(T)0) const; //Frobenius norm of A - scalar I
-        CSRMat & operator=(const T &a);          //assign a to diagonal
+	const T dot(const CSRMat &rhs) const; //\sum_{ij} conj(a_ij) b_ij as NRMat::dot
-        CSRMat & operator+=(const T &a);         //assign a to diagonal
+	const T dot(const NRMat<T> &rhs) const;
-        CSRMat & operator-=(const T &a);         //assign a to diagonal
+	const T trace() const;
-	void axpy(const T alpha, const CSRMat &x, const bool transp=0); // this+= a*x
+	void simplify(const typename LA_traits<T>::normtype sparseepsilon=LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()); //removes elements with |x| <= sparseepsilon
-        inline CSRMat & operator+=(const CSRMat &rhs) {axpy((T)1,rhs); return *this;};
+	inline const NRMat<T> operator*(const NRMat<T> &rhs) const {NRMat<T> r(nn,rhs.ncols()); r.gemm((T)0,*this,'n',rhs,'n',(T)1); return r;};
-        inline CSRMat & operator-=(const CSRMat &rhs) {axpy((T)-1,rhs); return *this;};
+	void gemm(const T beta, const CSRMat &a, const char transa, const CSRMat &b, const char transb, const T alpha); //this = alpha op(A) op(B) + beta this, sparse result
-	const T* diagonalof(NRVec<T> &, const bool divide=0, bool cache=false) const; //get diagonal
+	inline const CSRMat operator*(const CSRMat &rhs) const {CSRMat r; r.gemm((T)0,*this,'n',rhs,'n',(T)1); return r;};
-	void gemv(const T beta, NRVec<T> &r, const char trans, const T alpha, const NRVec<T> &x) const;
+	CSRMat & oplusequal(const CSRMat &rhs); //direct sum
-	inline const NRVec<T> operator*(const NRVec<T> &rhs) const {NRVec<T> result(nn); this->gemv((T)0,result,'n',(T)1,rhs); return result;};
+	CSRMat & oplusequal(const NRMat<T> &rhs) {return oplusequal(CSRMat(rhs));}
-	typename LA_traits<T>::normtype norm(const T scalar=(T)0) const;
+	CSRMat & oplusequal(const NRSMat<T> &rhs) {return oplusequal(CSRMat(rhs));}
-        inline const CSRMat operator*(const CSRMat &rhs) const {CSRMat<T> r(nn,mm); r.gemm(0,*this,'n',rhs,'n',1); return r;}; 
+	const CSRMat oplus(const CSRMat &rhs) const {return CSRMat(*this).oplusequal(rhs);}
-	void gemm(const T beta, const CSRMat &a, const char transa, const CSRMat &b, const char transb, const T alpha); //this := alpha*op( A )*op( B ) + beta*this 
+	const CSRMat oplus(const NRMat<T> &rhs) const {return CSRMat(*this).oplusequal(rhs);}
-	inline SPMatindex length() {return nnz;};
+	const CSRMat oplus(const NRSMat<T> &rhs) const {return CSRMat(*this).oplusequal(rhs);}
-	void transposeme() const {laerror("in-place transposition not implemented for CSRMat");};
+	const CSRMat otimes(const CSRMat &rhs) const; //direct product, (A x B)_{i nB+k, j mB+l} = a_ij b_kl as NRMat::operator|
-	CSRMat transpose(bool conj=false) const {laerror("CSR transpose not implemented");}
+	const CSRMat otimes(const NRMat<T> &rhs) const {return otimes(CSRMat(rhs));}
-	void get(int fd, bool dimen, bool transp);
+	const CSRMat otimes(const NRSMat<T> &rhs) const {return otimes(CSRMat(rhs));}
-        void put(int fd, bool dimen, bool transp) const;
+	//permutations: element of row r goes to row p(r) (with inverse p^{-1}); matrix indices from 0, permutation from 1; the full matrix is stored, so rows means only rows
-	int nrows() const {return nn;}
+	void permuteme_rows(const NRPerm<int> &p, const bool inverse=false);
-	int ncols() const {return mm;}
+	void permuteme_cols(const NRPerm<int> &p, const bool inverse=false);
-
+	void permuteme_both(const NRPerm<int> &p, const NRPerm<int> &q, const bool inverse=false);
-	/*class iterator {//not efficient, just for output to ostreams
+	const CSRMat permuted_rows(const NRPerm<int> &p, const bool inverse=false) const {CSRMat a(*this); a.permuteme_rows(p,inverse); return a;};
-        private:
+	const CSRMat permuted_cols(const NRPerm<int> &p, const bool inverse=false) const {CSRMat a(*this); a.permuteme_cols(p,inverse); return a;};
-                matel<T> *p;
+	const CSRMat permuted_both(const NRPerm<int> &p, const NRPerm<int> &q, const bool inverse=false) const {CSRMat a(*this); a.permuteme_both(p,q,inverse); return a;};
-		matel<T> my;
+	void permuterows(const NRVec<SPMatindex> &p); //as SparseMat: row r goes to row p[r], indices from 0
-		SPMatindex mynn;
+	void permutecolumns(const NRVec<SPMatindex> &p);
-		SPMatindex mymm;
+	void permuteindices(const NRVec<SPMatindex> &p); //rows and columns
-		
+	void transposeme(bool conj=false) {*this=transpose(conj);}
-
+	void get(int fd, bool dimensions=1, bool transp=0, bool orcaformat=false);
-        public:
+	void put(int fd, bool dimensions=1, bool transp=0, bool orcaformat=false) const;
-		//compiler-generated iterator & operator=(const  iterator &rhs); 
-		//compiler-generated iterator(const iterator &rhs); 
-                iterator()
-                iterator(const CSRMat &rhs)
-                iterator operator++()  
-		iterator(matel<T> *q) {p=q; col=NULL;}//just for end()
-                //compiler-generated ~iterator() {};
-                bool operator!=(const iterator &rhs) const {return p!=rhs.p;} //only for comparison with end()
-                bool operator==(const iterator &rhs) const {return p==rhs.p;} //only for comparison with end()
-                matel<T> & operator*() const {return *p;}
-                matel<T> * operator->() const {return p;}
-		bool notend() const {return (bool)p;};
-        };
-        iterator begin() const {return iterator(*this);}
-        iterator end() const {return iterator(NULL);}
-	*/
 };
 
 template<typename T>
-std::ostream & operator<<(std::ostream &s, const CSRMat<T> &x);
+inline const CSRMat<T> commutator(const CSRMat<T> &x, const CSRMat<T> &y, const bool trx=0, const bool tryy=0)
+{
+CSRMat<T> r;
+r.gemm((T)0,x,trx?'t':'n',y,tryy?'t':'n',(T)1);
+r.gemm((T)1,y,tryy?'t':'n',x,trx?'t':'n',(T)-1);
+return r;
+}
+
+template<typename T>
+inline const CSRMat<T> anticommutator(const CSRMat<T> &x, const CSRMat<T> &y, const bool trx=0, const bool tryy=0)
+{
+CSRMat<T> r;
+r.gemm((T)0,x,trx?'t':'n',y,tryy?'t':'n',(T)1);
+r.gemm((T)1,y,tryy?'t':'n',x,trx?'t':'n',(T)1);
+return r;
+}
+
+template <typename T>
+inline const NRVec<T> NRVec<T>::operator*(const CSRMat<T> &mat) const
+{
+NOT_GPU(*this);
+NRVec<T> result(mat.ncols());
+result.gemv((T)0,mat,'t',(T)1,*this);
+return result;
+}
 
 template <class T>
-std::istream& operator>>(std::istream  &s, CSRMat<T> &x);
+std::ostream& operator<<(std::ostream &s, const CSRMat<T> &x) //the same text format as SparseMat
+		{
+		s << (int)x.nrows() << ' ' << (int)x.ncols() << '\n';
+		for(typename CSRMat<T>::iterator p=x.begin(); p.notend(); ++p)
+			s << (int)p->row << ' ' << (int)p->col << ' ' << (typename LA_traits_io<T>::IOtype)p->elem << '\n';
+		s << "-1 -1\n";
+		return s;
+		}
 
+template <class T>
+std::istream& operator>>(std::istream &s, CSRMat<T> &x)
+		{
+		SparseMat<T> a;
+		s >> a;
+		x=CSRMat<T>(a);
+		return s;
+		}
 
 }//namespace
 #endif //_CSRMAT_H_

davidson.h

@@ -47,7 +47,7 @@ namespace LA {
 
 template <typename T, typename Matrix>
 void davidson(const Matrix &bigmat, NRVec<T> &eivals, NRVec<T> *eivecs, const char *eivecsfile, 
-		int nroots=1,  const bool verbose=0, const double eps=1e-6,
+		int nroots=1,  const bool verbose=0, const double eps=LA_precision<typename LA_traits<T>::normtype>::eigensolver_eps(),
 	 	const bool incore=1, int maxit=100, const int maxkrylov = 500,
 		void (*initguess)(NRVec<T> &)=NULL, const typename LA_traits<T>::normtype *target=NULL)
 {
@@ -235,7 +235,7 @@ eival_n = r[nroot];
 for(i=0; i<n; ++i)
 	{
 	typename LA_traits<T>::normtype denom = LA_traits<T>::realpart(diagonal[i]) - eival_n;
-        denom = denom<0?-std::max(0.1,std::abs(denom)):std::max(0.1,std::abs(denom));
+        denom = denom<0?-std::max((typename LA_traits<T>::normtype)0.1,std::abs(denom)):std::max((typename LA_traits<T>::normtype)0.1,std::abs(denom));
         vec1[i] /= denom;
 	}
 }

doc/bib/mat.tex

@@ -0,0 +1,5 @@
+%bibliography items of the chapter NRMat (mat.tex), cited in the comments of strassen.cc
+\bibitem{mat:strassen1969} V. Strassen, Numer. Math. \textbf{13}, 354 (1969).
+\bibitem{mat:winograd1971} S. Winograd, Linear Algebra Appl. \textbf{4}, 381 (1971).
+\bibitem{mat:boyer2009} B. Boyer, J.-G. Dumas, C. Pernet, W. Zhou, in \emph{Proceedings of ISSAC 2009}, ACM, New York 2009, p.~55.
+\bibitem{mat:husslederman1996} S. Huss-Lederman, E. M. Jacobson, J. R. Johnson, A. Tsao, T. Turnbull, in \emph{Proceedings of Supercomputing '96}, IEEE, 1996.

doc/bib/sparse.tex

@@ -0,0 +1,2 @@
+%bibliography items of the chapter Sparse matrices (sparse.tex), cited in the comments of csrmat.cc
+\bibitem{sparse:gustavson1978} F. G. Gustavson, ACM Trans. Math. Softw. \textbf{4}, 250 (1978).

doc/building.tex

@@ -5,7 +5,7 @@
 \begin{itemize}
 \item a C++11 capable compiler (the library is developed with GNU g++),
 \item BLAS and LAPACK (Fortran interface); optionally the C interfaces CBLAS and CLAPACK,
-	or Intel MKL, or ATLAS,
+	or Intel MKL, or OpenBLAS, or ATLAS,
 \item GNU autotools (autoconf, automake, libtool) when building from the git repository,
 \item optionally NVIDIA CUBLAS (experimental, section~\ref{sec:concepts:gpu}),
 \item optionally doxygen for the generated source code reference, which is independent of this manual.
@@ -16,7 +16,7 @@
 \begin{lstlisting}[language=bash]
 autoreconf --install       # only if ./configure does not exist yet
 ./configure [options] CXXFLAGS=-I<path> LDFLAGS=-L<path>
-make                       # library, and this manual and the doxygen pages if pdflatex/doxygen were found
+make                       # library, and this manual if pdflatex was found
 make check                 # builds the test programs, runs testoptimize
 make install
 \end{lstlisting}
@@ -29,6 +29,10 @@ accordingly:
 \begin{enumerate}
 \item Intel MKL (\code{mkl_intel_lp64}, \code{mkl_sequential}, \code{mkl_core} and the header
 	\header{mkl_cblas.h}); if found, \code{-DHAS_MKL -DNONCLAPACK} is used;
+\item otherwise OpenBLAS, if \code{libopenblas} contains LAPACK (\code{dgeev_}) and CBLAS (\code{cblas_dgemm}): only
+	\code{-lopenblas} is linked, \code{-DNONCLAPACK} is used and \code{-DNONCBLAS} if \header{cblas.h} is missing; skipped
+	with \code{--without-openblas} and with \code{--enable-fortran64int}. A threaded OpenBLAS uses all cores unless
+	\code{OPENBLAS_NUM_THREADS} is set;
 \item otherwise ATLAS (optional) and the reference BLAS (mandatory), CBLAS (if \code{cblas_ddot} and \header{cblas.h}
 	are not found, \code{-DNONCBLAS} is set) and LAPACK (mandatory; if the C interface \code{clapack_dgesv}
 	with \header{clapack.h} is not found, \code{-DNONCLAPACK} is set);
@@ -47,7 +51,8 @@ accordingly:
 \code{--enable-optimize=<flags>} & compiler optimization flags, default \code{-O3 -finline-limit=1000}; \code{--disable-optimize} for none \\
 \code{--enable-profile} & compile with \code{-pg} \\
 \code{--enable-matptr} & define \code{MATPTR}: \code{NRMat} stores an array of row pointers (\code{T**}), see chapter~\ref{ch:mat}; incompatible with CUDA \\
-\code{--enable-fortran64int} & define \code{LONG_FORTRAN_INT} to link with BLAS and LAPACK using 64-bit integers; implies \code{NONCBLAS} and \code{NONCLAPACK} (the old misspelling \code{--enable-fotran64int} is accepted as an alias) \\
+\code{--enable-fortran64int} & define \code{LONG_FORTRAN_INT} to link with BLAS and LAPACK using 64-bit integers; implies \code{NONCBLAS} and \code{NONCLAPACK} (the old misspelling \code{--enable-fotran64int} is accepted as an alias); OpenBLAS is then not used \\
+\code{--without-openblas} & do not use OpenBLAS even if found (ATLAS or the reference BLAS is used) \\
 \bottomrule
 \end{longtable}
 
@@ -72,7 +77,6 @@ functions do not match.
 \code{LONG_FORTRAN_INT} & Fortran integers are 64-bit (\code{long}) \\
 \code{MATPTR} & row pointer representation of \code{NRMat} \\
 \code{CUDALA} & GPU support via CUBLAS (experimental) \\
-\code{NO_STRASSEN} & do not compile the experimental Strassen matrix multiplication interface (\header{strassen.cc}, needs an external library); always set in \header{Makefile.am} \\
 \bottomrule
 \end{longtable}
 
@@ -117,5 +121,5 @@ This manual is in the subdirectory \header{doc}. \code{make manual} (alias \code
 is done only when a \code{.tex} file is newer than the PDF, and a \LaTeX{} error stops the build with the end of the log.
 A source code reference is generated by doxygen with the configuration file \header{doxygen.cfg} by \code{make doxygen}
 (output in \header{doxygen-doc}, rebuilt when a source file changes); it is independent of this manual.
-\code{make} (i.e.\ \code{make all}) builds both in addition to the library, each only if \code{configure} found
+\code{make} (i.e.\ \code{make all}) builds the manual in addition to the library if \code{configure} found
-\code{pdflatex} resp.\ \code{doxygen}; \code{make clean} removes \header{doc/.build} and \header{doxygen-doc} (not the PDF).
+\code{pdflatex}; the doxygen pages are built only by \code{make doxygen}. \code{make clean} removes \header{doc/.build} and \header{doxygen-doc} (not the PDF).

doc/concepts.tex

@@ -8,7 +8,7 @@ it and describe only what is specific to the class.
 \subsection{Shared data}
 
 An object of \code{NRVec}, \code{NRMat}, \code{NRSMat} and of all other classes holding large data
-(\code{SparseMat}, \code{SparseSMat}, \code{Tensor}, \code{fourindex}, \code{Polynomial}, the permutation
+(\code{SparseMat}, \code{SparseSMat}, \code{CSRMat}, \code{Tensor}, \code{fourindex}, \code{Polynomial}, the permutation
 classes, \code{bitvector}, \dots) consists of a small header (dimensions, a pointer to the data and a pointer
 to a reference counter) and the data block. The copy constructor and the assignment operator between objects
 of the same type copy only the header and increment the reference counter, so that both objects share the
@@ -145,6 +145,7 @@ Its main members are:
 \lafunc{static void put(int fd, const T &x, ...); static void get(int fd, T &x, ...); multiput; multiget}{binary I/O, recursive for nested types}
 \lafunc{static T conjugate(const T &x); realpart(x); imagpart(x); setrealpart(x,y); setimagpart(x,y)}{complex helpers, trivial for real types}
 \lafunc{static bool bigger(const T &x, const T &y); smaller(x,y)}{comparison used for sorting; for complex types it is an error}
+\lafunc{static int strassen_cutoff()}{scalar types only: default \code{cutoff} of \code{NRMat::strassen} (section~\ref{sec:mat:strassen}), currently 2048 for all types}
 \end{lareference}
 The auxiliary template \code{isscalar<T>} distinguishes scalars from classes, \code{LA_traits_complex<T>}
 gives for a complex type the component type and the corresponding real vector and matrix types (used by
@@ -153,12 +154,36 @@ the mixed real--complex functions such as \code{gemv} of a complex vector with a
 The functions \code{LA::real(c)} and \code{LA::imag(c)} return references to the real and imaginary part of a
 \code{std::complex}, which can be used as l-values.
 
+Default thresholds of the numerical methods depend on the floating point precision and are collected in
+\code{LA_precision<T>} (\header{la_traits.h}), which is defined for \code{T = double} and \code{float} only (any other type
+gives a compile-time error); functions with an argument of type \code{T} use \code{LA_precision<typename LA_traits<T>::normtype>},
+so that complex types take the values of their real component. The double values are those used by the library before.
+\begin{center}
+\begin{tabular}{llll}
+\toprule
+member & double & float & used by \\
+\midrule
+\code{svd_threshold()} & $10^{-12}$ & $10^{-5}$ & \code{svd_solve}, \code{Tensor::Tucker} \\
+\code{taylor_precision()} & $10^{-14}$ & $10^{-7}$ & \code{exp}, \code{sincos}, \code{exptimes} (\header{matexp.h}) \\
+\code{conjgrad_tol()} & $10^{-8}$ & $10^{-5}$ & \code{conjgrad} \\
+\code{gmres_eps()} & $10^{-7}$ & $10^{-5}$ & \code{gmres} \\
+\code{eigensolver_eps()} & $10^{-6}$ & $10^{-4}$ & \code{davidson}, \code{lanczos} \\
+\code{newton_threshold()} & $10^{-14}$ & $10^{-6}$ & \code{Polynomial::newton} \\
+\code{det_tiny()} & $10^{-300}$ & $10^{-37}$ & underflow guard of the determinant in \code{linear_solve} \\
+\code{sparse_epsilon()} & $10^{-15}$ & $10^{-7}$ & zero elements of \code{SparseMat}, \code{SparseSMat} (\code{SPARSEEPSILON}) \\
+\code{cholesky_epsilon()} & $10^{-16}$ & $10^{-8}$ & pivots skipped by \code{SparseSMat::cholesky} (\code{CHOLESKYEPSILON}) \\
+\bottomrule
+\end{tabular}
+\end{center}
+
 \subsection{Element types and instantiation}
 
 The class templates are explicitly instantiated in the library for \code{double}, \code{std::complex<double>}
 and the integer types \code{char}, \code{short}, \code{int}, \code{long}, \code{long long} and their unsigned
-variants. The BLAS and LAPACK backed functions are implemented for \code{double} and \code{std::complex<double>}
+variants; \code{NRVec}, \code{NRMat}, \code{NRSMat}, \code{SparseMat}, \code{SparseSMat}, \code{Tensor}, \code{fourindex} (with \code{unsigned char} indices) and \code{Polynomial} also for \code{float} and
-(some also for mixed real--complex arguments); calling them for other types results in a run-time
+\code{std::complex<float>} (single precision support, 2026, in progress for the other classes); \code{CSRMat} only for the four floating point types. The BLAS and LAPACK backed
+functions are implemented for \code{double} and \code{std::complex<double>} (some also for mixed real--complex arguments)
+and for their single precision counterparts, which always use the single precision BLAS/LAPACK routines (\code{s}/\code{c}); calling them for other types results in a run-time
 \code{laerror} (``not implemented for this type'') or a link error. Functions implemented in the headers work
 for any type with the necessary arithmetic operations.
 

doc/fourindex.tex

@@ -24,9 +24,9 @@ Indices are opaque to the sparse classes (they are never compared with the dimen
 1-based indexing are possible there; the dense classes use indices from 1.
 
 Only \code{fourindex<unsigned char,double>}, \code{fourindex_ext<unsigned char,double>} and
-\code{fourindex_dense<nosymmetry,double,unsigned char>} are explicitly instantiated in fourindex.cc;
+\code{fourindex_dense<nosymmetry,double,unsigned char>} and the same with \code{float} are explicitly instantiated in fourindex.cc;
 everything else is implemented in the header and instantiated on use, except \code{fscanf()} and
-\code{fprintf()} which exist only for \code{fourindex<unsigned char,double>}.
+\code{fprintf()} which exist only for \code{fourindex<unsigned char,double>} and \code{fourindex<unsigned char,float>}.
 
 \section{Symmetry types}\label{sec:fourindex:symmetry}
 
@@ -131,7 +131,7 @@ The symmetry must be set (not \code{undefined_symmetry}) for \code{piterator} an
 \lafunc{unsigned long get(int fd, bool withattr=true)}{read the same format until end of file, \emph{adding} the elements to the list; returns their number.}
 \lafunc{template <class I, class T> std::ostream& operator<<(std::ostream &s, const fourindex<I,T> &x)}{text output: the dimension, lines \code{i j k l value}, and a final line with four terminators and 0.}
 \lafunc{template <class I, class T> std::istream& operator>>(std::istream &s, fourindex<I,T> &x)}{read the text format; \code{x} is resized to the read dimension (the symmetry is not set) and reading stops at the first line containing the terminator of \code{x} as an index.}
-\lafunc{void fscanf(FILE *f); void fprintf(FILE *f, char *format) const}{C-style text I/O, only for \code{fourindex<unsigned char,double>}; line format \code{value i j k l}. \code{fscanf} adds elements until a line in which any index equals the terminator (compared after conversion to the index type, so $-1$ in the file matches the default terminator 255); \code{fprintf} applies \code{format} (which must consume a \code{double} and four \code{int}s) to each element and ends with a line \code{0 t t t t} with the terminator $t$. With terminator 0 this is the FCIDUMP-like convention (blocks separated by zero lines, read by consecutive \code{fscanf} calls; a one-electron line $i\,j\,0\,0$ also ends a block).}
+\lafunc{void fscanf(FILE *f); void fprintf(FILE *f, char *format) const}{C-style text I/O, only for \code{fourindex<unsigned char,double>} and \code{<unsigned char,float>}; line format \code{value i j k l}. \code{fscanf} adds elements until a line in which any index equals the terminator (compared after conversion to the index type, so $-1$ in the file matches the default terminator 255); \code{fprintf} applies \code{format} (which must consume a \code{double} and four \code{int}s) to each element and ends with a line \code{0 t t t t} with the terminator $t$. With terminator 0 this is the FCIDUMP-like convention (blocks separated by zero lines, read by consecutive \code{fscanf} calls; a one-electron line $i\,j\,0\,0$ also ends a block).}
 \end{lareference}
 
 \section{Disc storage: \texttt{fourindex\_ext<I,T>}}\label{sec:fourindex:ext}

doc/introduction.tex

@@ -48,7 +48,7 @@ the Numerical Recipes library. They can be used with it, but are completely inde
 \header{mat.h} & \code{NRMat<T>}, \code{NRMat_from1<T>} (chapter~\ref{ch:mat}) \\
 \header{smat.h} & \code{NRSMat<T>}, \code{NRSMat_from1<T>} (chapter~\ref{ch:smat}) \\
 \header{sparsemat.h}, \header{sparsesmat.h} & sparse matrices \code{SparseMat<T>}, \code{SparseSMat<T>} (chapter~\ref{ch:sparse}) \\
-\header{csrmat.h} & compressed row storage sparse matrix (unfinished) \\
+\header{csrmat.h} & compressed row storage sparse matrix \code{CSRMat<T>} (chapter~\ref{ch:sparse}) \\
 \header{nonclass.h}, \header{matexp.h} & linear systems, eigenvalue problems, SVD, matrix functions (chapter~\ref{ch:nonclass}) \\
 \header{davidson.h}, \header{lanczos.h}, \header{conjgrad.h}, \header{gmres.h}, \header{diis.h}, \header{bisection.h} & iterative methods, DIIS convergence acceleration (chapter~\ref{ch:iterative}) \\
 \header{optimize.h} & numerical optimization (chapter~\ref{ch:optimize}) \\

doc/iterative.tex

@@ -14,6 +14,9 @@ or of the normal equations $A\Tr Ax=A\Tr b$ for rectangular $A$ (least squares);
 \item the class \code{DIIS} (\header{diis.h}): Pulay's DIIS convergence acceleration of a general fixed-point iteration;
 \item \code{bisection_find()} and \code{interpolation_find()} (\header{bisection.h}): search in a sorted array.
 \end{itemize}
+All of them work for \code{T = float} and \code{std::complex<float>} as well (where the double versions support the
+corresponding real or complex type); their default thresholds then come from \code{LA_precision<float>}
+(section~\ref{sec:concepts:traits}).
 Dense (LAPACK based) diagonalization, linear solvers and matrix functions are in chapter~\ref{ch:nonclass}; the small
 subspace problems of \code{davidson} and \code{lanczos} are solved by the routines described there
 (\code{diagonalize}, \code{gdiagonalize}, \code{diagonalize3}, \code{linear_solve}). The quasi-Newton GDIIS optimizer built on
@@ -96,7 +99,7 @@ problem.
 \laheader{davidson.h}
 
 \begin{lareference}
-\lafunc{template <typename T, typename Matrix> void davidson(const Matrix &bigmat, NRVec<T> &eivals, NRVec<T> *eivecs, const char *eivecsfile, int nroots=1, const bool verbose=0, const double eps=1e-6, const bool incore=1, int maxit=100, const int maxkrylov = 500, void (*initguess)(NRVec<T> &)=NULL, const typename LA_traits<T>::normtype *target=NULL)}{
+\lafunc{template <typename T, typename Matrix> void davidson(const Matrix &bigmat, NRVec<T> &eivals, NRVec<T> *eivecs, const char *eivecsfile, int nroots=1, const bool verbose=0, const double eps=LA_precision<normtype>::eigensolver_eps(), const bool incore=1, int maxit=100, const int maxkrylov = 500, void (*initguess)(NRVec<T> &)=NULL, const typename LA_traits<T>::normtype *target=NULL)}{
 Davidson's method \cite{iterative:davidson1975} for the \code{nroots} lowest eigenvalues of \code{bigmat}.
 \begin{itemize}
 \item \code{eivals}: on return the eigenvalues in its first \code{nroots} elements (ascending, or by increasing distance from
@@ -167,7 +170,7 @@ implicitly given one, built column by column by \code{gemv} on unit vectors (for
 \laheader{lanczos.h}
 
 \begin{lareference}
-\lafunc{template <typename T, typename Matrix> void lanczos(const Matrix &bigmat, NRVec<T> &eivals, NRVec<T> *eivecs, const char *eivecsfile, int nroots=1, const bool verbose=0, const double eps=1e-6, const bool incore=1, int maxit=100, const int maxkrylov = 1000, void (*initguess)(NRVec<T> &)=NULL, const typename LA_traits<T>::normtype *target=NULL)}{
+\lafunc{template <typename T, typename Matrix> void lanczos(const Matrix &bigmat, NRVec<T> &eivals, NRVec<T> *eivecs, const char *eivecsfile, int nroots=1, const bool verbose=0, const double eps=LA_precision<normtype>::eigensolver_eps(), const bool incore=1, int maxit=100, const int maxkrylov = 1000, void (*initguess)(NRVec<T> &)=NULL, const typename LA_traits<T>::normtype *target=NULL)}{
 Lanczos method for the \code{nroots} lowest (or closest to \code{*target}) eigenvalues of a hermitian matrix. The parameters
 have the same meaning as for \code{davidson}, with these differences: \code{bigmat.issymmetric()} must be \code{true}
 (\code{laerror} otherwise); the start vector is normalized by the function; \code{incore=false} keeps only the Lanczos vectors
@@ -194,7 +197,7 @@ the method fails with \code{laerror} when more than \code{maxit} iterations are
 \laheader{conjgrad.h}
 
 \begin{lareference}
-\lafunc{template<typename T, typename Matrix> bool conjgrad(const Matrix &bigmat, const NRVec<T> &b, NRVec<T> &x, const bool doguess=true, const double tol=1e-8, const int itmax=1000, const bool verbose=true, bool issquare=1, const bool precondition=1)}{
+\lafunc{template<typename T, typename Matrix> bool conjgrad(const Matrix &bigmat, const NRVec<T> &b, NRVec<T> &x, const bool doguess=true, const double tol=LA_precision<normtype>::conjgrad_tol(), const int itmax=1000, const bool verbose=true, bool issquare=1, const bool precondition=1)}{
 Solves $Ax=b$ by the (Jacobi preconditioned) conjugate gradient method (the source refers to the book of Stoer and Bulirsch).
 Returns \code{true} if converged, \code{false} after \code{itmax}+1 iterations without convergence; \code{x} then holds the
 last iterate.
@@ -227,7 +230,7 @@ $p\cdot Ap$ (or $|Ap|^2$) or the residual product $r\cdot z$ vanishes. Only real
 \laheader{gmres.h}
 
 \begin{lareference}
-\lafunc{template<typename T, typename Matrix> bool gmres(const Matrix &bigmat, const NRVec<T> &b, NRVec<T> &x, const bool doguess=1, const double eps=1e-7, const int MAXIT=50, const bool verbose=1, bool square=1, const bool precondition=1, int neustart=0, const int incore=1)}{
+\lafunc{template<typename T, typename Matrix> bool gmres(const Matrix &bigmat, const NRVec<T> &b, NRVec<T> &x, const bool doguess=1, const double eps=LA_precision<normtype>::gmres_eps(), const int MAXIT=50, const bool verbose=1, bool square=1, const bool precondition=1, int neustart=0, const int incore=1)}{
 Solves $Ax=b$ by restarted GMRES with Givens rotations (after H.~R.~Schwarz, original implementation by M.~Warken). Returns
 \code{true} if converged; otherwise \code{false}, with \code{x} holding the last iterate.
 \begin{itemize}

doc/mat.tex

@@ -30,9 +30,9 @@ reference count is larger than one (controlled by \code{_LA_count_check}); read
 possibly shared matrix through a const reference.
 
 \paragraph{Element types.} \code{mat.cc} explicitly instantiates \code{NRMat<T>} for
-\code{double}, \code{std::complex<double>}, and the integer types \code{char}, \code{short}, \code{int},
+\code{double}, \code{float}, \code{std::complex<double>}, \code{std::complex<float>}, and the integer types \code{char}, \code{short}, \code{int},
 \code{long}, \code{long long} and their unsigned variants. Many numerical members are only
-specialized (BLAS-backed) for \code{double} and \code{complex<double>} and have no generic
+specialized (BLAS-backed) for the four floating point types and have no generic
 definition, so using them with an integer type results in a link error. Table~\ref{tab:mat:types}
 summarizes this. Nested element types (e.g.\ \code{NRMat<NRVec<double> >}) are supported by the
 reference counting (type-nested \code{copyonwrite}, sec.~\ref{sec:concepts:traits}), but several
@@ -47,10 +47,10 @@ Members & Available for \\
 \midrule
 \texttt{gemm}, \texttt{norm}, \texttt{axpy(T,const NRMat\&)}, \texttt{dot}, \texttt{transpose}, \texttt{diagmultl}, \texttt{diagmultr},
 \texttt{operator*(const NRSMat\&)}, \texttt{diagonalset}, \texttt{inverse}, \texttt{svdinverse}, \texttt{absadjoint},
-\texttt{randomize}, \texttt{amax}, \texttt{amin} & \texttt{double}, \texttt{complex<double>} \\
+\texttt{randomize}, \texttt{amax}, \texttt{amin}, \texttt{strassen} & \texttt{double}, \texttt{float}, \texttt{complex<double>}, \texttt{complex<float>} \\
-\texttt{operator*(const NRMat\&)} & \texttt{double}, \texttt{complex<double>}, \texttt{int} \\
+\texttt{operator*(const NRMat\&)} & the same and \texttt{int} \\
-\texttt{diagonalof}, \texttt{orthonormalize}, \texttt{strassen}, \texttt{gemm} with \texttt{SparseMat} & \texttt{double} only \\
+\texttt{diagonalof}, \texttt{orthonormalize}, \texttt{gemm} with \texttt{SparseMat} & \texttt{double}, \texttt{float} \\
-\texttt{nonhermiticity} & \texttt{complex<double>} only \\
+\texttt{nonhermiticity} & \texttt{complex<double>}, \texttt{complex<float>} \\
 everything else & all instantiated types \\
 \bottomrule
 \end{tabular}
@@ -73,14 +73,23 @@ MATPTR is rarely used and less tested than the default representation; the libra
 regression test (\code{testoptimize}).
 
 \paragraph{Strassen multiplication.}\label{sec:mat:strassen}
-Unless \code{NO_STRASSEN} is defined, \code{NRMat} declares \code{strassen()} and \code{s_cutoff()}
+\code{strassen()} (in \code{strassen.cc}, for \code{double}, \code{float}, \code{complex<double>} and \code{complex<float>}) computes the
-(specialized for \code{double} only, in \code{strassen.cc}). They call the external Fortran-compatible
+same $C=\beta C+\alpha\,\mathrm{op}(A)\,\mathrm{op}(B)$ as \code{gemm} with the same parameters, plus an optional \code{cutoff}, by the
-routines \code{fmm} and \code{strassen_cutoff}, which are \emph{not} part of \LA{}
+algorithm of Strassen \cite{mat:strassen1969} in the variant of Winograd (7 products and 15 additions of quadrants per level)
-(the test code in \code{t.cc} mentions linking \code{-lstrassen}). Strassen multiplication is never
+\cite{mat:winograd1971}, which needs $O(n^{\log_2 7})=O(n^{2.81})$ operations. The recursion works on strided quadrants of the original
-used implicitly: \code{operator*} and \code{gemm} always call BLAS \code{gemm}. Note that
+matrices (no copies of $A$ and $B$, transpositions are passed to the products), the products of the blocks are computed by BLAS
-\code{Makefile.am} unconditionally adds \code{-DNO_STRASSEN}, so in the standard build these members do not exist.
+\code{gemm}, the order of operations follows the memory-saving schedule with two temporaries per level of \cite{mat:boyer2009} and odd
-\lawarning{The Strassen interface is \emph{experimental}: it has not been used or tested for a long time. It is kept for possible future work,
+dimensions are treated by dynamic peeling \cite{mat:husslederman1996}. A level of recursion is done only while all of $m$, $n$, $k$ are larger
-in particular if fast (sub-$O(n^3)$) matrix multiplication algorithms become practically useful.}
+than \code{cutoff} (default \code{LA_traits<T>::strassen_cutoff()}, 2048 for all types), so products with a dimension $\le$
+\code{cutoff} are plain \code{gemm} calls. Workspace of about $\frac23 mn$ elements is allocated (plus $mn$ if $\beta\ne0$).
+Strassen multiplication is never used implicitly: \code{operator*} and \code{gemm} always use the classical algorithm.
+
+Whether it pays off depends on the BLAS and the machine: on top of an optimized multithreaded BLAS the saving of one product in eight
+is largely eaten by the memory-bound additions, so a gain is to be expected mainly for large matrices ($n\gtrsim4000$) and single-threaded
+BLAS; the cutoff should be tuned by a benchmark. CPU only (\code{laerror} for matrices on the GPU); the device-dependent operations
+are separated in \code{strassen.cc} for a future CUDA version.
+\lawarning{The error of Strassen multiplication is bounded only normwise, not componentwise as for the classical algorithm, and the bound
+grows by roughly a factor of 12 per level of recursion. Small elements of the result may therefore be less accurate than with \code{gemm}.}
 
 \section{Construction, assignment, conversion}\label{sec:mat:constr}
 \begin{lareference}
@@ -93,7 +102,7 @@ in particular if fast (sub-$O(n^3)$) matrix multiplication algorithms become pra
 \lafunc{explicit NRMat(const int n) = delete}{A single integer argument is forbidden (it would otherwise silently create a $3\times3$ matrix
 through \code{Mat3}); use \code{NRMat(n,m)}.}
 \lafunc{NRMat(const NRMat &rhs)}{Shallow copy (shares data).}
-\lafunc{NRMat(const typename LA_traits_complex<T>::NRMat_Noncomplex_type &rhs, bool imagpart = false)}{Complexifying (non-explicit) constructor, defined for \code{T = complex<double>}: \code{rhs} (\code{NRMat<double>}) becomes the real part (or the imaginary part if \code{imagpart}), the other part is zero.}
+\lafunc{NRMat(const typename LA_traits_complex<T>::NRMat_Noncomplex_type &rhs, bool imagpart = false)}{Complexifying (non-explicit) constructor, defined for \code{T = complex<double>} and \code{complex<float>}: \code{rhs} (\code{NRMat<double>} resp.\ \code{NRMat<float>}) becomes the real part (or the imaginary part if \code{imagpart}), the other part is zero.}
 \lafunc{explicit NRMat(const NRMat<std::complex<T> > &rhs, bool imagpart = false)}{Decomplexifying constructor, defined for \code{T = double}: real part (or imaginary part if \code{imagpart}) of \code{rhs}.}
 \lafunc{explicit NRMat(const NRSMat<T> &rhs)}{Full square matrix from a packed symmetric one ($A_{ij}=A_{ji}$; for complex types hermitian, $A_{ij}=A_{ji}^*$).}
 \lafunc{explicit NRMat(const NRVec<T> &rhs, const int n, const int m, const int offset = 0)}{View of the vector data from position \code{offset} as an $n\times m$ matrix. Without MATPTR the data are \emph{shared} (reference count of the vector is incremented); with MATPTR they are copied. A nonzero offset gives a view into the middle of the vector without copying (a deliberate feature);
@@ -102,7 +111,7 @@ its condition of use is that the vector outlives the matrix, see Pitfalls. The c
 \lafunc{explicit NRMat(const WeightPermutation<int,T> &p, const bool direction)}{Weight times permutation matrix.}
 \lafunc{explicit NRMat(const PermutationAlgebra<int,T> &p, const bool direction, const int nforce = 0)}{Sum of weighted permutation matrices of the elements of a permutation algebra; size taken from the first element unless \code{nforce>0}. (The header remarks that the permutation-matrix representation of $S_n$ is reducible to only the irreps $[n]$ and $[n-1,1]$, so e.g.\ Young projectors cannot be represented this way.) See chapter~\ref{ch:permutation}.}
 \lafunc{explicit NRMat(const SparseMat<T> &rhs); explicit NRMat(const SparseSMat<T> &rhs); explicit NRMat(const SemiSparseMat<T> &rhs)}{Dense matrix from sparse ones (chapter~\ref{ch:sparse}).}
-\lafunc{explicit NRMat(const CSRMat<T> &rhs)}{Not supported: \code{CSRMat} is unfinished, the declaration is kept for possible future work (no definition, a link error if used).}
+\lafunc{explicit NRMat(const CSRMat<T> &rhs)}{Dense matrix from a compressed row sparse one (section~\ref{sec:sparse:csr}).}
 \lafunc{~NRMat()}{Decrements the reference count, frees the data when it reaches zero.}
 \lafunc{NRMat & operator=(const NRMat &rhs)}{Shallow assignment (sec.~\ref{sec:concepts:assign}).}
 \lafunc{NRMat & operator|=(const NRMat &rhs)}{Deep copy.}
@@ -140,10 +149,11 @@ Note that scalar addition and subtraction act on the \emph{diagonal} only, i.e.\
 \lafunc{NRMat & operator^=(const NRMat<T> &rhs)}{Hadamard (element-wise) product $A_{ij}\leftarrow A_{ij}B_{ij}$.}
 \lafunc{void axpy(const T alpha, const NRMat &x)}{$A\leftarrow A+\alpha X$ (BLAS \code{axpy}).}
 \lafunc{const NRMat operator*(const NRMat &rhs) const}{Matrix product $AB$ (BLAS \code{gemm}; simple loops for \code{int}).}
-\lafunc{void gemm(const T &beta, const NRMat &a, const char transa, const NRMat &b, const char transb, const T &alpha)}{$C\leftarrow\alpha\,\mathrm{op}(A)\,\mathrm{op}(B)+\beta C$ with $C$ = \code{*this}, which must already have the correct dimensions (it is not resized; if $\beta=0$ its contents may be uninitialized). \code{trans}: \code{'n'} no transposition; for \code{double} any other character means transposition; for \code{complex<double>} \code{'t'} transposition and \code{'c'} conjugate transposition (case insensitive). Returns immediately when $\alpha=0$ and $\beta=1$. \code{*this} must not share storage with \code{a} or \code{b}.}
+\lafunc{void gemm(const T &beta, const NRMat &a, const char transa, const NRMat &b, const char transb, const T &alpha)}{$C\leftarrow\alpha\,\mathrm{op}(A)\,\mathrm{op}(B)+\beta C$ with $C$ = \code{*this}, which must already have the correct dimensions (it is not resized; if $\beta=0$ its contents may be uninitialized). \code{trans}: \code{'n'} no transposition; for real types any other character means transposition; for complex types \code{'t'} transposition and \code{'c'} conjugate transposition (case insensitive). Returns immediately when $\alpha=0$ and $\beta=1$. \code{*this} must not share storage with \code{a} or \code{b}.}
-\lafunc{void gemm(const T &beta, const SparseMat<T> &a, const char transa, const NRMat &b, const char transb, const T &alpha)}{The same with a sparse first factor; \code{double} only (the complex version calls \code{laerror("not implemented yet")}), see chapter~\ref{ch:sparse}.}
+\lafunc{void gemm(const T &beta, const SparseMat<T> &a, const char transa, const NRMat &b, const char transb, const T &alpha)}{The same with a sparse first factor; \code{double} and \code{float} only (the complex versions call \code{laerror("not implemented yet")}), see chapter~\ref{ch:sparse}.}
 \lafunc{const NRMat operator*(const NRSMat<T> &rhs) const}{$AS$ with packed symmetric (real) or hermitian (complex) $S$ (BLAS \code{dspmv}/\code{zhpmv} per row).}
 \lafunc{SparseSMat<T> operator*(const SparseSMat<T> &rhs) const}{Dense times sparse, the result is sparse (chapter~\ref{ch:sparse}).}
+\lafunc{void gemm(const T &beta, const CSRMat<T> &a, const char transa, const NRMat &b, const char transb, const T &alpha); const NRMat operator*(const CSRMat<T> &rhs) const}{$C\leftarrow\beta C+\alpha\,\mathrm{op}(A)\,\mathrm{op}(B)$ with a compressed row sparse $A$ (all types, \code{trans} \code{'n'}, \code{'t'}, \code{'c'}), and dense times compressed row sparse with a dense result (section~\ref{sec:sparse:csr}).}
 \lafunc{const NRVec<T> operator*(const NRVec<T> &rhs) const; const NRVec<std::complex<T> > operator*(const NRVec<std::complex<T> > &rhs) const}{Matrix--vector product via \code{NRVec::gemv} (the second form: real matrix times complex vector).}
 \lafunc{void gemv(const T beta, NRVec<T> &r, const char trans, const T alpha, const NRVec<T> &x) const; void gemv(const T beta, NRVec<std::complex<T> > &r, const char trans, const T alpha, const NRVec<std::complex<T> > &x) const}{$r\leftarrow\alpha\,\mathrm{op}(A)x+\beta r$; forwards to \code{r.gemv(beta,*this,trans,alpha,x)}, see chapter~\ref{ch:vec}.}
 \lafunc{const T dot(const NRMat &rhs) const}{$\sum_{ij}A_{ij}^*B_{ij}$ (Frobenius inner product, conjugating \code{*this}).}
@@ -153,7 +163,7 @@ Note that scalar addition and subtraction act on the \emph{diagonal} only, i.e.\
 \lafunc{const NRMat operator&(const NRMat &rhs) const; const NRMat oplus(const NRMat &rhs) const}{Direct sum $A\oplus B$ (block-diagonal matrix). \code{oplus} returns the other operand when one is empty; \code{operator&} is plain-data only.}
 \lafunc{const NRMat operator|(const NRMat<T> &rhs) const}{Direct (Kronecker) product $A\otimes B$, $(A\otimes B)_{i n_B+k,\,j m_B+l}=A_{ij}B_{kl}$.}
 \lafunc{const NRMat otimes(const NRMat &rhs, bool reversecolumns = false) const}{Kronecker product as \code{operator|}; with \code{reversecolumns} the column index is ordered the other way, $R_{i n_B+k,\,l m_A+j}=A_{ij}B_{kl}$. Returns the empty operand when one is empty.}
-\lafunc{void strassen(const T beta, const NRMat &a, const char transa, const NRMat &b, const char transb, const T alpha); void s_cutoff(const int, const int, const int, const int) const}{Only without \code{NO_STRASSEN}, \code{double} only: Strassen multiplication with \code{gemm} semantics (dimensions always checked) and setting of the cutoffs of the external Strassen library (section~\ref{sec:mat:strassen}).}
+\lafunc{void strassen(const T &beta, const NRMat &a, const char transa, const NRMat &b, const char transb, const T &alpha, const int cutoff=LA_traits<T>::strassen_cutoff())}{Strassen-Winograd multiplication with the parameters and semantics of \code{gemm}; blocks with a dimension $\le$ \code{cutoff} are multiplied by \code{gemm} (section~\ref{sec:mat:strassen}).}
 \end{lareference}
 
 \section{Other member functions}\label{sec:mat:other}
@@ -167,13 +177,13 @@ Note that scalar addition and subtraction act on the \emph{diagonal} only, i.e.\
 \lafunc{const NRMat submatrix(const NRVec<int> &rows, const NRVec<int> &cols) const}{Copy of the elements $A_{r_i c_j}$ for given (arbitrary) lists of row and column indices; indices are always range-checked.}
 \lafunc{void storesubmatrix(const int fromrow, const int fromcol, const NRMat &rhs)}{Store \code{rhs} as a block with the upper left corner at (\code{fromrow},\code{fromcol}). Plain data only.}
 \lafunc{void storesubmatrix(const NRVec<int> &rows, const NRVec<int> &cols, const NRMat &rhs)}{Scatter: $A_{r_i c_j}\leftarrow R_{ij}$.}
-\lafunc{const T* diagonalof(NRVec<T> &r, const bool divide = 0, bool cache = false) const}{\code{double} only. For a square matrix copy the diagonal into \code{r}, or with \code{divide} divide \code{r} element-wise by the (nonzero) diagonal elements. For a non-square matrix the diagonal of $A\Tr A$ (squared column norms) is used instead; \code{r} must have \code{ncols()} elements. Returns pointer to \code{r}'s data, or NULL if \code{divide}. \code{cache} is unused. Same interface as in other matrix classes, so that it can be used by iterative solvers as a preconditioner (chapter~\ref{ch:iterative}).}
+\lafunc{const T* diagonalof(NRVec<T> &r, const bool divide = 0, bool cache = false) const}{\code{double} and \code{float} only. For a square matrix copy the diagonal into \code{r}, or with \code{divide} divide \code{r} element-wise by the (nonzero) diagonal elements. For a non-square matrix the diagonal of $A\Tr A$ (squared column norms) is used instead; \code{r} must have \code{ncols()} elements. Returns pointer to \code{r}'s data, or NULL if \code{divide}. \code{cache} is unused. Same interface as in other matrix classes, so that it can be used by iterative solvers as a preconditioner (chapter~\ref{ch:iterative}).}
 \lafunc{void diagonalset(const NRVec<T> &r)}{Set the diagonal; \code{r} has $\min(n,m)$ elements.}
 \end{lareference}
 
 \subsection{Transposition and conjugation}
 \begin{lareference}
-\lafunc{const NRMat transpose(bool conj = false) const}{$A\Tr$, or $A\Hc$ if \code{conj} (complex). \code{double} and \code{complex<double>} only.}
+\lafunc{const NRMat transpose(bool conj = false) const}{$A\Tr$, or $A\Hc$ if \code{conj} (complex). Floating point types only.}
 \lafunc{NRMat& transposeme(const int n = 0)}{In-place transposition of a square matrix (no conjugation); with \code{n>0} only the leading $n\times n$ block is transposed (this also works for a non-square matrix with $n\le\min(\text{rows},\text{cols})$). Dimensions are not changed, so a non-square matrix cannot be transposed in place.}
 \lafunc{NRMat& conjugateme(); const NRMat conjugate() const}{Complex conjugation in place resp.\ by value; no-op for real types.}
 \end{lareference}
@@ -198,7 +208,7 @@ Note that scalar addition and subtraction act on the \emph{diagonal} only, i.e.\
 \lafunc{const T sum() const; const NRVec<T> rsum() const; const NRVec<T> csum() const}{Sum of all elements; vector of row sums (length \code{nrows()}, sum over the column index); vector of column sums (length \code{ncols()}).}
 \lafunc{inline const T amax() const; inline const T amin() const}{The element (not its index) with the largest resp.\ smallest absolute value; for complex the measure is $|\mathrm{Re}|+|\mathrm{Im}|$ (BLAS \code{i?amax} convention).}
 \lafunc{const typename LA_traits<T>::normtype nonsymmetry() const}{$\sum_{i>j}|A_{ij}-A_{ji}|^2$ (square of the norm, no square root).}
-\lafunc{const typename LA_traits<T>::normtype nonhermiticity() const}{\code{complex<double>} only: $\sum_{i\ge j}|A_{ij}-A_{ji}^*|^2$.}
+\lafunc{const typename LA_traits<T>::normtype nonhermiticity() const}{Complex types only: $\sum_{i\ge j}|A_{ij}-A_{ji}^*|^2$.}
 \lafunc{NRMat<typename LA_traits<T>::normtype> abs() const}{Matrix of element-wise absolute values.}
 \lafunc{NRMat<typename LA_traits<T>::normtype> diffabs(const NRMat &rhs) const}{Element-wise $|A_{ij}|-|B_{ij}|$.}
 \lafunc{const bool operator==(const NRMat &rhs) const; const bool operator!=(const NRMat &rhs) const}{Equality of dimensions and contents; for plain types a bitwise (\code{memcmp}) comparison, element-wise for nested types.}
@@ -207,7 +217,7 @@ Note that scalar addition and subtraction act on the \emph{diagonal} only, i.e.\
 
 \subsection{Orthonormalization, inverse, random fill}
 \begin{lareference}
-\lafunc{void orthonormalize(const bool rowcol, const NRSMat<T> *metric = NULL)}{\code{double} only. Modified Gram--Schmidt orthonormalization of the rows (\code{rowcol==true}) or columns (\code{false}) of the matrix in place, optionally with respect to the inner product $x\Tr S y$ given by \code{metric}. A zero vector (or non-positive metric) raises \code{laerror}.}
+\lafunc{void orthonormalize(const bool rowcol, const NRSMat<T> *metric = NULL)}{\code{double} and \code{float} only. Modified Gram--Schmidt orthonormalization of the rows (\code{rowcol==true}) or columns (\code{false}) of the matrix in place, optionally with respect to the inner product $x\Tr S y$ given by \code{metric}. A zero vector (or non-positive metric) raises \code{laerror}.}
 \lafunc{NRMat inverse() const}{$A^{-1}$ via \code{calcinverse} (LAPACK linear solve, chapter~\ref{ch:nonclass}).}
 \lafunc{NRMat svdinverse(const typename LA_traits<T>::normtype thr = 0) const}{Pseudo-inverse via SVD, singular values below \code{thr} are dropped.}
 \lafunc{NRMat absadjoint(const typename LA_traits<T>::normtype thr = 0) const}{Adjugate (transposed cofactor) matrix computed via SVD up to the factor $\det U\det V$ (a sign, or a phase in the complex case), cf.\ \code{calcsvdinverse} in chapter~\ref{ch:nonclass}. \code{thr} is not used in this mode.}
@@ -229,6 +239,7 @@ See section~\ref{sec:concepts:io} for the general conventions.
 Besides the constructors, \code{operator*(const SparseSMat<T>&)} and \code{gemm} with a \code{SparseMat}
 factor listed above, \code{NRMat} declares
 \begin{lareference}
+\lafunc{NRMat & operator+=(const CSRMat<T> &rhs); NRMat & operator-=(const CSRMat<T> &rhs)}{Add/subtract a compressed row sparse matrix (section~\ref{sec:sparse:csr}).}
 \lafunc{NRMat & operator+=(const SparseMat<T> &rhs); NRMat & operator-=(const SparseMat<T> &rhs)}{Add/subtract the sparse matrix (defined in \code{sparsemat.h}; a symmetric \code{SparseMat} is unfolded, without conjugation).}
 \end{lareference}
 
@@ -322,6 +333,6 @@ NRMat<double> ds = re.oplus(im);          // direct sum, 4x4 block diagonal
 \lawarning{Writing through \code{operator[]}, \code{operator()} or \code{operator T*()} into a matrix obtained by copy construction or \code{=} modifies all sharing matrices unless \code{copyonwrite()} is called first. In DEBUG builds even \emph{reading} through the non-const operators of a shared matrix raises an error; use a const reference.}
 \lawarning{\code{NRMat(const NRVec<T>&, n, m, offset)} shares the vector data (without MATPTR). With a nonzero offset, if the matrix happens to be the last owner of the data, it will \code{delete[]} a pointer that is not the beginning of the allocated block; the constructor therefore prints a warning. This is by design (a view without copying, avoiding the cost of a copy); the condition of use is: keep the vector alive longer than the
 matrix, or make an independent copy (\code{|=}) if the matrix has to outlive it.}
-\lawarning{Many numerical members (\code{gemm}, \code{transpose}, \code{norm}, \ldots) exist only for \code{double} and \code{complex<double>} (Table~\ref{tab:mat:types}); with other element types the program fails to link.}
+\lawarning{Many numerical members (\code{gemm}, \code{transpose}, \code{norm}, \ldots) exist only for the floating point types \code{double}, \code{float}, \code{complex<double>} and \code{complex<float>} (Table~\ref{tab:mat:types}); with other element types the program fails to link.}
 \lawarning{\code{transposeme()} does not conjugate and works only on square matrices (or square leading blocks); use \code{transpose(true)} for $A\Hc$ or for a non-square matrix.}
 \lawarning{In the non-DEBUG build no dimension checks are done in most operations (\code{operator+=}, \code{gemm}, \code{operator=(const T&)}, \ldots); mismatches lead to silent memory corruption.}

doc/nonclass.tex

@@ -6,8 +6,8 @@ This chapter describes the free (non-member) functions operating on the dense cl
 \code{NRVec}, \code{NRMat} and \code{NRSMat}: linear systems, determinants, inverses, Cholesky
 decomposition, symmetric/hermitian and general eigenvalue problems, singular value decomposition,
 matrix functions, and a few auxiliary products and conversions. Most of them are thin wrappers of
-LAPACK routines (\header{nonclass.cc}); they exist for \code{T}${}={}$\code{double} and
+LAPACK routines (\header{nonclass.cc}); they exist for \code{T}${}={}$\code{double}, \code{float},
-\code{std::complex<double>} only. The templates in \header{matexp.h} (matrix polynomials, matrix
+\code{std::complex<double>} and \code{std::complex<float>} only. The templates in \header{matexp.h} (matrix polynomials, matrix
 exponential, $\exp(A)\,x$, BCH expansion) are generic: they work for every class that provides the few
 operations listed in section~\ref{sec:nonclass:matexp}, including sparse matrices
 (chapter~\ref{ch:sparse}) and user-defined matrix classes which only implement \code{gemv}.
@@ -44,8 +44,8 @@ growing leading block.
 (sec.~\ref{sec:concepts:errors}), e.g.\ ``singular matrix'' or ``convergence problem''. Exceptions:
 \code{linear_solve_x} and \code{multiply_by_inverse} return \code{info}.
 
-\paragraph{Element types and availability.} The functions are declared for \code{double} and
+\paragraph{Element types and availability.} The functions are declared for \code{double}, \code{float},
-\code{std::complex<double>} by the macro \code{declare_la}, but not all declared complex variants are
+\code{std::complex<double>} and \code{std::complex<float>} by the macro \code{declare_la}, but not all declared complex variants are
 defined in \header{nonclass.cc}. Calling an undefined one gives a link error
 (table~\ref{tab:nonclass:avail}).
 \begin{table}[htbp]
@@ -54,15 +54,15 @@ defined in \header{nonclass.cc}. Calling an undefined one gives a link error
 \toprule
 Function & Available for \\
 \midrule
-\texttt{linear\_solve(NRMat\&, NRMat*, ...)} & \texttt{double}, \texttt{complex<double>} \\
+\texttt{linear\_solve(NRMat\&, NRMat*, ...)} & \texttt{double}, \texttt{float}, \texttt{complex<double>}, \texttt{complex<float>} \\
-\texttt{linear\_solve(NRMat\&, NRVec\&, ...)}, \texttt{linear\_solve(NRSMat\&, ...)} & \texttt{double}, \texttt{complex<double>} (complex \texttt{NRSMat}: hermitian) \\
+\texttt{linear\_solve(NRMat\&, NRVec\&, ...)}, \texttt{linear\_solve(NRSMat\&, ...)} & \texttt{double}, \texttt{float}, \texttt{complex<double>}, \texttt{complex<float>} (complex \texttt{NRSMat}: hermitian) \\
 \texttt{determinant}, \texttt{determinant\_destroy} & \texttt{NRMat}, \texttt{NRSMat}, real and complex \\
 \texttt{diagonalize}, \texttt{singular\_decomposition}, \texttt{cholesky}, \texttt{diagofproduct},
-\texttt{trace2(NRMat,NRMat)}, \texttt{linear\_solve\_x}, \texttt{multiply\_by\_inverse}, \texttt{MatrixNorm}, \texttt{CondNumber} & \texttt{double}, \texttt{complex<double>} \\
+\texttt{trace2(NRMat,NRMat)}, \texttt{linear\_solve\_x}, \texttt{multiply\_by\_inverse}, \texttt{MatrixNorm}, \texttt{CondNumber} & \texttt{double}, \texttt{float}, \texttt{complex<double>}, \texttt{complex<float>} \\
 \texttt{gdiagonalize} & real matrix (real or complex eigenvector storage), complex matrix \\
-\texttt{trace2} with \texttt{NRSMat} & \texttt{double}, \texttt{complex<double>} (hermitian) \\
+\texttt{trace2} with \texttt{NRSMat} & \texttt{double}, \texttt{float}, \texttt{complex<double>}, \texttt{complex<float>} (hermitian) \\
 \texttt{diagonalize3}, \texttt{realmatrixfunction}, \texttt{complexmatrixfunction},
-\texttt{adjustphases}, \texttt{xcopy}, \texttt{xaxpy}, \texttt{xdot} & \texttt{double} only \\
+\texttt{adjustphases}, \texttt{xcopy}, \texttt{xaxpy}, \texttt{xdot} & \texttt{double}, \texttt{float} \\
 templates of \texttt{matexp.h}, \texttt{power}, \texttt{commutator}, \texttt{twoside\_transform}, \dots & any type providing the needed operations \\
 \bottomrule
 \end{tabular}
@@ -106,27 +106,27 @@ $n_{\mathrm{rhs}}=0$).}
 \lafunc{void linear_solve(NRSMat<T> &a, NRMat<T> *b, T *det=0, int n=0); void linear_solve(NRSMat<T> &a, NRVec<T> &b, T *det=0, int n=0)}{Symmetric (real) or hermitian (complex), possibly indefinite $A$ in packed storage, \code{dspsv}/\code{zhpsv}
 (Bunch--Kaufman factorization; the determinant is computed from its $1\times1$ and $2\times2$ blocks and is real in the hermitian case).
 \code{n} selects a leading block, which is contiguous in the packed storage.}
-\lafunc{int linear_solve_x(NRMat<T> &A, T *B, const int rhsCount, const int eqCount, const bool eq, const bool saveA, double *rcond)}{Expert solver
+\lafunc{int linear_solve_x(NRMat<T> &A, T *B, const int rhsCount, const int eqCount, const bool eq, const bool saveA, typename LA_traits<T>::normtype *rcond)}{Expert solver
 (\code{dgesvx}/\code{zgesvx}): solves $A x=b$ for \code{rhsCount} right-hand sides stored consecutively (each of length \code{eqCount}) in the raw
 array \code{B}, using the leading \code{eqCount}${}\times{}$\code{eqCount} block of \code{A}. \code{eq}: equilibrate $A$ (for badly conditioned matrices);
 then \code{A} is overwritten unless \code{saveA}. The reciprocal condition number is stored in \code{*rcond} if non-NULL. Returns LAPACK \code{info}
 (0 success, $>0$ singular or ill-conditioned). The solution replaces \code{B}.}
-\lafunc{int multiply_by_inverse(NRMat<T> &A, NRMat<T> &B, bool useEq=false, double *rcond=NULL)}{$B\leftarrow A B^{-1}$ for square $A$, $B$ via
+\lafunc{int multiply_by_inverse(NRMat<T> &A, NRMat<T> &B, bool useEq=false, typename LA_traits<T>::normtype *rcond=NULL)}{$B\leftarrow A B^{-1}$ for square $A$, $B$ via
 \code{dgesvx}/\code{zgesvx}; \code{useEq} as \code{eq} above. \code{A} is not modified (with \code{useEq} a temporary copy is
 scaled). Returns \code{info}.}
 \lafunc{const NRMat<T> calcinverse(NRMat<T> a, T *det=NULL)}{$A^{-1}$ by \code{linear_solve} with unit right-hand sides; optionally the determinant.
 \code{NRMat::inverse()} (chapter~\ref{ch:mat}) calls it.}
-\lafunc{const NRMat<T> calcsvdinverse(NRMat<T> a, double thr=0, bool do_absadjoint=false)}{Inverse via SVD, $V\,\mathrm{diag}(1/s_i)\,U\Hc$,
+\lafunc{const NRMat<T> calcsvdinverse(NRMat<T> a, typename LA_traits<T>::normtype thr=0, bool do_absadjoint=false)}{Inverse via SVD, $V\,\mathrm{diag}(1/s_i)\,U\Hc$,
 singular values $s_i<$\code{thr} are treated as zero (pseudoinverse). With \code{do_absadjoint} returns
 $V\,\mathrm{diag}(\prod_{j\ne i}s_j)\,U\Hc$, the adjugate matrix up to the phase $\det U\det V$. Square matrices only.}
-\lafunc{NRVec<T> svd_solve(NRMat<T> &a, const NRVec<T> &b, double thres=1e-12, int *nullity=NULL); NRMat<T> svd_solve(NRMat<T> &a, const NRMat<T> &b, double thres=1e-12, int *nullity=NULL)}{Least-squares
+\lafunc{NRVec<T> svd_solve(NRMat<T> &a, const NRVec<T> &b, normtype thres=LA_precision<normtype>::svd_threshold(), int *nullity=NULL); NRMat<T> svd_solve(NRMat<T> &a, const NRMat<T> &b, normtype thres=LA_precision<normtype>::svd_threshold(), int *nullity=NULL)}{Least-squares
 solution $x=V\,\mathrm{diag}(1/s_i)\,U\Hc b$ discarding singular values $\le$\code{thres} (no contamination by the null space);
 the number of discarded ones is stored in \code{*nullity}. Here (unlike \code{linear_solve}) the right-hand sides are the
 \emph{columns} of the matrix \code{b} ($m\times n_{\mathrm{rhs}}$); \code{a} ($m\times n$, $m\ge n$) is destroyed.}
 \lafunc{const LA_traits<MAT>::elementtype determinant(MAT a)}{Determinant by LU (\code{NRMat}) or Bunch--Kaufman (\code{NRSMat}) factorization of a copy.}
 \lafunc{const LA_traits<MAT>::elementtype determinant_destroy(MAT &a)}{The same, overwriting \code{a}. Unless \code{HAS_MKL} is defined, a dummy right-hand side
 is passed to \code{linear_solve} (work-around for OpenBLAS, which skips the factorization without right-hand sides).}
-\lafunc{void cholesky(NRMat<double> &a, bool upper=1); void cholesky(NRMat<std::complex<double> > &a, bool upper=1)}{Cholesky factorization of a
+\lafunc{void cholesky(NRMat<T> &a, bool upper=1)}{Cholesky factorization of a
 positive definite matrix in place (\code{dpotrf}/\code{zpotrf}). \code{upper=1}: \code{a} becomes the upper triangular $U$ with
 $A=U\Hc U$; \code{upper=0}: lower triangular $L$ with $A=L L\Hc$. The other triangle is zeroed. Raises \code{laerror} if $A$ is not positive definite.}
 \end{lareference}
@@ -138,15 +138,15 @@ Eigenvalues are always returned in \textbf{ascending order} (LAPACK). Only the \
 (row index ${}\ge{}$ column index) of an \code{NRMat} is referenced (verified); \code{NRSMat} stores only that
 triangle and is interpreted as hermitian (sec.~\ref{sec:smat:hermitian}).
 \begin{lareference}
-\lafunc{void diagonalize(NRMat<T> &a, NRVec<double> &w, const bool eivec=1, const bool corder=1, int n=0, NRMat<T> *b=NULL, const int itype=1)}{Eigenvalues
+\lafunc{void diagonalize(NRMat<T> &a, NRVec<typename LA_traits<T>::normtype> &w, const bool eivec=1, const bool corder=1, int n=0, NRMat<T> *b=NULL, const int itype=1)}{Eigenvalues
 \code{w} (size must equal \code{a.nrows()}) and, if \code{eivec}, eigenvectors stored in \code{a} (columns for \code{corder=1}, see table~\ref{tab:nonclass:eivec})
 of a real symmetric or complex hermitian matrix (\code{dsyev}/\code{zheev}). If \code{b} is given, the generalized problem with positive definite $B$
 is solved (\code{dsygv}/\code{zhegv}): \code{itype=1}: $A x=\lambda B x$, \code{itype=2}: $A B x=\lambda x$, \code{itype=3}: $B A x=\lambda x$;
 eigenvectors are $B$-normalized ($x\Hc B x=1$ for \code{itype} 1, 2), \code{b} is overwritten by its Cholesky factor.}
-\lafunc{void diagonalize(NRSMat<T> &a, NRVec<double> &w, NRMat<T> *v, const bool corder=1, int n=0, NRSMat<T> *b=NULL, const int itype=1)}{The same for packed
+\lafunc{void diagonalize(NRSMat<T> &a, NRVec<typename LA_traits<T>::normtype> &w, NRMat<T> *v, const bool corder=1, int n=0, NRSMat<T> *b=NULL, const int itype=1)}{The same for packed
 storage (\code{dspev}/\code{zhpev}, \code{dspgv}/\code{zhpgv}); eigenvectors go to \code{*v} (square, at least $n\times n$) or are not computed if \code{v==NULL}.
 \code{a} (and \code{b}) are destroyed.}
-\lafunc{void diagonalize3(NRVec<double> &d, NRVec<double> &d1, NRMat<double> *v, const bool corder=1, int n0=0)}{Real symmetric tridiagonal matrix
+\lafunc{void diagonalize3(NRVec<double> &d, NRVec<double> &d1, NRMat<double> *v, const bool corder=1, int n0=0)}{(also for \code{float}) Real symmetric tridiagonal matrix
 (\code{dsteqr}) with diagonal \code{d} and sub-diagonal \code{d1} (size ${}\ge n-1$). On return \code{d} holds the eigenvalues (ascending), \code{d1} is destroyed,
 \code{*v} (exactly $n\times n$) the eigenvectors. \code{n0>0} uses only the first \code{n0} elements. Used by \code{lanczos} (chapter~\ref{ch:iterative}).}
 \end{lareference}
@@ -158,7 +158,7 @@ may contain zeros for infinite eigenvalues). Right (\code{vr}) and left (\code{v
 pointers are non-NULL; storage according to table~\ref{tab:nonclass:eivec}. Without sorting, the eigenvalues come in
 LAPACK order (unsorted, complex conjugate pairs adjacent with positive imaginary part first).
 \begin{lareference}
-\lafunc{void gdiagonalize(NRMat<double> &a, NRVec<double> &wr, NRVec<double> &wi, NRMat<double> *vl=NULL, NRMat<double> *vr=NULL, const bool corder=1, int n=0, const int sorttype=0, const int biorthonormalize=0, NRMat<double> *b=NULL, NRVec<double> *beta=NULL)}{Real
+\lafunc{void gdiagonalize(NRMat<double> &a, NRVec<double> &wr, NRVec<double> &wi, NRMat<double> *vl=NULL, NRMat<double> *vr=NULL, const bool corder=1, int n=0, const int sorttype=0, const int biorthonormalize=0, NRMat<double> *b=NULL, NRVec<double> *beta=NULL)}{(also for \code{float}) Real
 matrix, eigenvalues $w_r+\mathrm{i}\,w_i$, eigenvectors in the real storage of \code{dgeev}: for a real eigenvalue the vector is real; for a complex pair
 $(j,j{+}1)$ the vectors $j$ and $j{+}1$ hold the real and imaginary part. Verified: the right eigenvector belonging to
 $w_{r,j}+\mathrm{i}w_{i,j}$ ($w_{i,j}>0$) is $\mathrm{Re}-\mathrm{i}\,\mathrm{Im}$, and $\mathrm{Re}+\mathrm{i}\,\mathrm{Im}$ of \code{vl} satisfies $y\Tr A=\lambda y\Tr$ --
@@ -167,15 +167,15 @@ note that this sign is opposite to the \code{dgeev} documentation.
 2 real eigenvalues first, then complex ones, each group ordered as for 1. Eigenvectors are permuted accordingly.
 \code{biorthonormalize} (needs \code{vl} and \code{vr}, not with \code{b}): 1 scales the left, 2 the right eigenvectors so that $y_k\Tr x_k=1$;
 complex pairs are rotated to biorthonormality. \code{a} is destroyed (and used as scratch space).}
-\lafunc{void gdiagonalize(NRMat<T> &a, NRVec<std::complex<double> > &w, NRMat<std::complex<double> > *vl=NULL, NRMat<std::complex<double> > *vr=NULL, const bool corder=1, int n=0, const int sorttype=0, const int biorthonormalize=0, NRMat<T> *b=NULL, NRVec<T> *beta=NULL)}{Template
+\lafunc{void gdiagonalize(NRMat<T> &a, NRVec<typename LA_traits<T>::complextype> &w, NRMat<typename LA_traits<T>::complextype> *vl=NULL, NRMat<typename LA_traits<T>::complextype> *vr=NULL, const bool corder=1, int n=0, const int sorttype=0, const int biorthonormalize=0, NRMat<T> *b=NULL, NRVec<T> *beta=NULL)}{Template
-specialized for \code{T=double} (calls the previous routine and converts to complex eigenvalues and eigenvectors) and \code{T=complex<double>} (\code{zgeev}/\code{zggev}).
+specialized for \code{T=double}, \code{float} (calls the previous routine and converts to complex eigenvalues and eigenvectors) and \code{T=complex<double>}, \code{complex<float>} (\code{zgeev}/\code{zggev}, \code{cgeev}/\code{cggev}).
 For the complex matrix, \code{sorttype>0} raises \code{laerror} (not implemented) and \code{biorthonormalize} scales so that $\bar y_k\Tr x_k=1$ in the row storage.}
-\lafunc{void gdiagonalize(NRMat<std::complex<double> > &a, NRVec<double> &wr, NRVec<double> &wi, NRMat<std::complex<double> > *vl=NULL, NRMat<std::complex<double> > *vr=NULL, const bool corder=1, int n=0, const int sorttype=0, const int biorthonormalize=0, NRMat<std::complex<double> > *b=NULL, NRVec<std::complex<double> > *beta=NULL)}{Complex
+\lafunc{void gdiagonalize(NRMat<std::complex<double> > &a, NRVec<double> &wr, NRVec<double> &wi, NRMat<std::complex<double> > *vl=NULL, NRMat<std::complex<double> > *vr=NULL, const bool corder=1, int n=0, const int sorttype=0, const int biorthonormalize=0, NRMat<std::complex<double> > *b=NULL, NRVec<std::complex<double> > *beta=NULL)}{(also for \code{complex<float>}) Complex
 matrix with real and imaginary parts of the eigenvalues in separate vectors; exists to make \code{davidson} compile for complex \code{T}.}
 \end{lareference}
 \section{Singular value decomposition}\label{sec:nonclass:svd}
 \begin{lareference}
-\lafunc{void singular_decomposition(NRMat<T> &a, NRMat<T> *u, NRVec<double> &s, NRMat<T> *v, const bool vnotdagger=0, int m=0, int n=0)}{SVD
+\lafunc{void singular_decomposition(NRMat<T> &a, NRMat<T> *u, NRVec<typename LA_traits<T>::normtype> &s, NRMat<T> *v, const bool vnotdagger=0, int m=0, int n=0)}{SVD
 $A=U\,\mathrm{diag}(s)\,V\Hc$ of the $m\times n$ matrix \code{a} (\code{dgesvd}/\code{zgesvd}; \code{m}, \code{n} select a leading block, default whole matrix).
 \code{s} (size ${}\ge\min(m,n)$) receives the singular values in descending order. \code{u}: \code{NULL} (not computed), $m\times m$ (all columns) or
 $m\times\min(m,n)$ (economy size, chosen by \code{u} being non-square). \code{v}: \code{NULL}, $n\times n$ or $\min(m,n)\times n$, receives $V\Hc$
@@ -186,11 +186,12 @@ $m\times\min(m,n)$ (economy size, chosen by \code{u} being non-square). \code{v}
 Functions of matrices computed via the eigendecomposition $f(A)=V f(\Lambda) V^{-1}$. For general matrices
 the eigenvalue problem is solved in complex arithmetic and the result converted back to the element type of the
 argument; for a real argument the imaginary part is \emph{silently dropped} (its size is not checked). The matrix must be
-diagonalizable; nothing is done about defective or ill-conditioned eigenvector matrices.
+diagonalizable; nothing is done about defective or ill-conditioned eigenvector matrices. The functions listed below with
+\code{double} in the signature exist also with \code{float} (and \code{complex<float>} in place of \code{complex<double>}).
 \begin{lareference}
-\lafunc{NRMat<T> matrixfunction(NRSMat<T> a, double (*f)(double))}{$f(A)$ for a real symmetric or complex hermitian packed matrix, real function \code{f}
+\lafunc{NRMat<T> matrixfunction(NRSMat<T> a, typename LA_traits<T>::normtype (*f)(typename LA_traits<T>::normtype))}{$f(A)$ for a real symmetric or complex hermitian packed matrix, real function \code{f}
 of the (real) eigenvalues.}
-\lafunc{NRMat<T> matrixfunction(NRMat<T> a, std::complex<double> (*f)(const std::complex<double> &))}{$f(A)$ for a general real or complex
+\lafunc{NRMat<T> matrixfunction(NRMat<T> a, typename LA_traits<T>::complextype (*f)(const typename LA_traits<T>::complextype &))}{$f(A)$ for a general real or complex
 matrix by \code{gdiagonalize}, $f(A)=\sum_k f(\lambda_k)\,x_k y_k\Tr/(y_k\Tr x_k)$.}
 \lafunc{NRMat<double> realmatrixfunction(NRMat<double> a, double (*f)(double))}{$f(A)$ for a real symmetric matrix in full storage (only the lower triangle is used).}
 \lafunc{NRMat<std::complex<double> > complexmatrixfunction(NRMat<double> a, double (*fre)(double), double (*fim)(double))}{Complex function
@@ -223,8 +224,8 @@ so they can be used with implicitly defined (``direct'') matrices as in chapter~
 needs \code{size()}, \code{operator*=}, \code{axpy} and \code{resize()}.
 
 The exponential uses scaling and squaring: the power $p$ is chosen so that $\|A\|/2^p\lesssim 0.75$
-(\code{mynextpow2}), the Taylor series of the scaled argument is truncated when the next term would be below $10^{-14}$
+(\code{mynextpow2}), the Taylor series of the scaled argument is truncated when the next term would be below
-relative to the norm, and the result is squared $p$ times. The scaling is put into the Taylor coefficients,
+\code{LA_precision<normtype>::taylor_precision()} ($10^{-14}$ for double, $10^{-7}$ for float) relative to the norm, and the result is squared $p$ times. The scaling is put into the Taylor coefficients,
 so the argument itself is not copied. \code{maxpower} (if ${}\ge0$) limits $p$, \code{maxtaylor} (if ${}\ge0$) limits
 the order of the Taylor polynomial, which is useful and exact for nilpotent matrices (e.g.\ coupled-cluster excitation
 operators).
@@ -285,7 +286,7 @@ with two \code{gemm} calls; \code{trx}, \code{tryy} transpose the factors (no co
 \section{Norms and condition number}\label{sec:nonclass:norms}
 \begin{lareference}
 \lafunc{typename LA_traits<MAT>::normtype MatrixNorm(const MAT &A, const char norm)}{Matrix norm by \code{dlange}/\code{zlange}, implemented for
-\code{NRMat<double>} and \code{NRMat<std::complex<double> >} (explicit specializations of the template, which can be added for other types).
+\code{NRMat} of \code{double}, \code{float}, \code{std::complex<double>} and \code{std::complex<float>} (explicit specializations of the template, which can be added for other types).
 \code{'1'} or \code{'O'}: 1-norm (maximum column sum); \code{'I'}: $\infty$-norm (maximum row sum); \code{'F'} or \code{'E'}: Frobenius norm;
 \code{'M'}: largest absolute value of an element (lower case accepted). Works for non-square matrices.}
 \lafunc{typename LA_traits<MAT>::normtype CondNumber(const MAT &A, const char norm)}{Condition number $\|A\|\,\|A^{-1}\|$ of a square matrix in the 1-norm
@@ -296,10 +297,10 @@ infinity for a singular matrix. It is LAPACK's estimate, a lower bound that is u
 \section{Real and imaginary parts}\label{sec:nonclass:complex}
 \begin{lareference}
 \lafunc{const LA_traits<T>::realtype realpart(const T &a); const LA_traits<T>::realtype imagpart(const T &a)}{Real or imaginary part as a real object;
-specialized for \code{NRVec}, \code{NRMat}, \code{NRSMat} and \code{SparseMat} over \code{complex<double>}.}
+specialized for \code{NRVec}, \code{NRMat}, \code{NRSMat}, \code{SparseMat} and \code{CSRMat} over \code{complex<double>} and \code{complex<float>}.}
 \lafunc{const LA_traits<T>::complextype realmatrix(const T &a); const LA_traits<T>::complextype imagmatrix(const T &a)}{Complex object $A$ resp.\ $\mathrm{i}A$ from a real one;
-for \code{NRMat<double>} and \code{SparseSMat<double>}.}
+for \code{NRMat}, \code{SparseSMat} and \code{CSRMat} of \code{double} and \code{float}.}
-\lafunc{const LA_traits<T>::complextype complexmatrix(const T &re, const T &im)}{Complex object $\mathrm{re}+\mathrm{i}\,\mathrm{im}$; for \code{NRMat<double>} and \code{SparseSMat<double>}.}
+\lafunc{const LA_traits<T>::complextype complexmatrix(const T &re, const T &im)}{Complex object $\mathrm{re}+\mathrm{i}\,\mathrm{im}$; for \code{NRMat}, \code{SparseSMat} and \code{CSRMat} of \code{double} and \code{float}.}
 \end{lareference}
 For the same purpose the classes provide \code{complexify()} (chapters~\ref{ch:vec}, \ref{ch:mat}).
 The real and imaginary part functions are CPU only (sec.~\ref{sec:concepts:gpu}).
@@ -314,7 +315,7 @@ Application from the source: overlap of Slater determinants with signed spin-orb
 \lafunc{NRMat<T> reconstructmatrix<M,T>(const M &implicitmat)}{Explicit matrix of an implicit one (applying \code{gemv} to unit vectors), for debugging;
 both template arguments must be given. Equivalent to \code{explicit_matrix<T>} of \header{davidson.h} (chapter~\ref{ch:iterative}).}
 \lafunc{void xcopy(int n, const T *x, int incx, T *y, int incy); void xaxpy(int n, const T &a, const T *x, int incx, T *y, int incy); T xdot(int n, const T *x, int incx, const T *y, int incy)}{Type-generic
-names of the CBLAS routines (to ease switching between \code{float}/\code{double}/complex in user code); at present specialized only for \code{double}.}
+names of the CBLAS routines (to ease switching between \code{float}/\code{double}/complex in user code); specialized for \code{double} and \code{float}.}
 \end{lareference}
 
 \section{Examples}\label{sec:nonclass:examples}

doc/optimize.tex

@@ -23,12 +23,13 @@ Methods are written on top of a small infrastructure (evaluation counter, iterat
 common drivers (line search, first order, trust region), so that a new method usually consists of a short class supplying the
 search direction, the update rule or the trust-region step (section~\ref{sec:optimize:drivers}).
 
-All templates are parameterized by the real type \code{T}, but optimize.cc instantiates them for \code{T = double} only.
+All templates are parameterized by the real type \code{T}; optimize.cc instantiates them for \code{T = double} and
-The capability interfaces, adapters and the hyperparameter classes are header-only templates; the optimizers, the adapters with
+\code{float}. The capability interfaces, adapters and the hyperparameter classes are header-only templates; the optimizers, the
-member functions in optimize.cc (\code{FrozenOptProblem}, \code{ParameterShift}, \code{FiniteDifference},
+adapters with member functions in optimize.cc (\code{FrozenOptProblem}, \code{ParameterShift}, \code{FiniteDifference},
-\code{OptLeastSquares}) and the auxiliary functions exist in the library only for \code{double}.
+\code{OptLeastSquares}) and the auxiliary functions exist in the library for \code{double} and \code{float}.
 The common drivers \code{linesearch_minimize}, \code{firstorder_minimize} and \code{trustregion_minimize}, the extension
-points for user-written methods, are explicitly instantiated for \code{double} as well.
+points for user-written methods, are explicitly instantiated for both types as well. The defaults depending on the precision
+are taken from \code{opt_precision<T>} (section~\ref{sec:optimize:hyper}).
 
 The data members of all classes of this chapter are small, except vectors held by \code{FrozenOptProblem} and
 \code{optevaluator}; these are ordinary \code{NRVec} objects following the reference counting and copy-on-demand scheme of
@@ -254,6 +255,29 @@ hyperparameters                    maxstep(200) maxfunc(0) f_thr(0) x_thr(0) sad
  +- Cobyla_hyperparameters         rhobeg(0.5) rhoend(1e-6) flexible_radius(true); maxstep=100000
 \end{lstlisting}
 
+The defaults given above and below are those for \code{double}. The precision-dependent ones are taken from the trait
+\code{opt_precision<T>} (\header{optimize.h}, analogous to \code{LA_precision<T>} of section~\ref{sec:concepts:traits}), defined
+for \code{T = double} and \code{float} only; the double values are those used by the library before. In float the gradient norm
+reachable by a line search is limited by the resolution of $f$ (about $\epsilon|f|$ divided by the step), hence the larger
+\code{g_thr}.
+\begin{center}
+\begin{tabular}{llll}
+\toprule
+member & double & float & default of \\
+\midrule
+\code{fd_central_step()} & $10^{-6}$ & $5\cdot10^{-3}$ & \code{hh} of \code{FiniteDifference}, \code{ParameterShift} \\
+\code{fd_forward_step()} & $10^{-7}$ & $3\cdot10^{-4}$ & \code{h} of \code{OptLeastSquares}, \code{my_fd_step} (LM) \\
+\code{g_thr()} & $10^{-6}$ & $10^{-3}$ & \code{my_g_thr} \\
+\code{sy_thr()} & $10^{-12}$ & $10^{-6}$ & \code{my_sy_thr} (quasi-Newton) \\
+\code{sr1_thr()} & $10^{-8}$ & $10^{-4}$ & \code{my_sr1_thr} (SR1) \\
+\code{update_thr()} & $10^{-8}$ & $10^{-4}$ & \code{my_update_thr} (trust region) \\
+\code{nm_f_thr()}, \code{nm_x_thr()} & $10^{-8}$, $10^{-6}$ & $10^{-5}$, $10^{-4}$ & \code{my_f_thr}, \code{my_x_thr} (Nelder-Mead) \\
+\code{powell_f_thr()}, \code{powell_x_thr()} & $10^{-10}$, $10^{-6}$ & $10^{-6}$, $10^{-4}$ & \code{my_f_thr}, \code{my_x_thr} (Powell) \\
+\code{rhoend()} & $10^{-6}$ & $10^{-4}$ & \code{my_rhoend} (COBYLA) \\
+\bottomrule
+\end{tabular}
+\end{center}
+
 \paragraph{Base classes.}
 \begin{lareference}
 \lafunc{template<typename T> class hyperparameters}{Parameters common to all methods.}
@@ -671,16 +695,19 @@ saddle order.}
 
 For least-squares problems providing \code{OptResiduals} (typically derived from \code{OptLeastSquares}): the step $h$ solves
 $(J\Tr J+\mu D)h=-J\Tr r$ (\code{linear_solve}), $D=I$ or $D=\mathrm{diag}(J\Tr J)$ (Marquardt scaling); the damping starts at
-$\mu=\tau\max_i(J\Tr J)_{ii}$ and is updated by the gain ratio $\varrho$ \cite{optimize:madsen2004} (alg.~3.16): for $\varrho>0$
+$\mu=\tau\max_i(J\Tr J)_{ii}$ (with scaling $\mu=\tau$, the same relative to $D$) and is updated by the gain ratio $\varrho$ \cite{optimize:madsen2004} (alg.~3.16): for $\varrho>0$
 the step is accepted and $\mu\leftarrow\mu\max(1/3,1-(2\varrho-1)^3)$, $\nu=2$, otherwise $\mu\leftarrow\mu\nu$, $\nu\leftarrow2\nu$.
 The Jacobian comes from \code{jacobian()} of the problem or from forward differences (relative step \code{my_fd_step}, $n$
 residual evaluations counted in \code{my_maxfunc}). Convergence: $|J\Tr r|<$ \code{my_g_thr}, the general $f$/$x$ criterion, or
-when a rejected step becomes negligible at machine precision ($|h|\le10\epsilon(|x|+\epsilon)$). The function value returned is
+when a rejected step becomes negligible at machine precision ($h_i^2\le(10\epsilon)^2(x_i^2+\epsilon^2)$ for all $i$,
+tested per component to be scale invariant, cf.\ \cite{optimize:madsen2004} eq.~3.16). The function value returned is
 $f=\frac12|r|^2$. The problem's \code{funcgrad()} is not used.
 \begin{lareference}
 \lafunc{template<typename T> class LM_hyperparameters : public gradient_hyperparameters<T>}{}
 \lafunc{T my_tau}{Initial damping factor $\tau$ (\code{1e-3}).}
-\lafunc{bool my_scaling}{Marquardt scaling $D=\mathrm{diag}(J\Tr J)$ (default \code{false}, $D=I$).}
+\lafunc{bool my_scaling}{Marquardt scaling $D=\mathrm{diag}(J\Tr J)$ (default \code{false}, $D=I$). Recommended for
+badly scaled parameters, in float usually necessary (otherwise the steps in the weakly curved directions fall below the
+resolution of $f$).}
 \lafunc{T my_fd_step}{Relative step of the forward-difference Jacobian (\code{1e-7}).}
 \lafunc{template<typename T> class LM_Optimizer : public Optimizer<T>}{\code{laerror} if the problem does not provide
 \code{OptResiduals}. Verbose name Levenberg-Marquardt.}

doc/polynomial.tex

@@ -10,7 +10,7 @@ This chapter describes the polynomial class \code{Polynomial<T>} and the exact r
 \code{template <typename T> class Polynomial : public NRVec<T>} represents $p(x)=\sum_{i=0}^{d}c_ix^i$; the coefficient $c_i$ is the vector element
 \code{p[i]} (indexed from 0, i.e.\ by the power) and the vector has $d+1$ elements. The member \code{size()} is made private to prevent confusion; use
 \code{degree()}. The leading coefficient may be zero (e.g.\ after a subtraction); \code{simplify()} removes such terms. The class is explicitly
-instantiated for \code{int}, \code{unsigned int}, \code{double}, \code{std::complex<double>}; root finding and SVD-based gcd require a floating point type.
+instantiated for \code{int}, \code{unsigned int}, \code{double}, \code{std::complex<double>}, \code{float}, \code{std::complex<float>}; root finding and SVD-based gcd require a floating point type.
 
 All inherited \code{NRVec} functionality (norms, \code{randomize}, \code{clear}, I/O operators printing the number of coefficients followed by the
 coefficients, \ldots) is available. \code{Polynomial} follows the copy-on-demand scheme of section~\ref{sec:concepts:refcount}: its modifying members call
@@ -77,10 +77,10 @@ vector, hence also a \code{Polynomial} (section~\ref{sec:nonclass:matexp}).
 \subsection{Roots, interpolation, gcd}
 \begin{lareference}
 \lafunc{NRMat<T> companion() const}{companion matrix ($d\times d$, ones on the superdiagonal, last row $-c_i/c_d$) whose characteristic polynomial is $p/c_d$.}
-\lafunc{NRVec<typename LA_traits<T>::complextype> roots() const}{all roots as eigenvalues of the companion matrix (\code{gdiagonalize}, chapter~\ref{ch:nonclass}); for \code{double}
+\lafunc{NRVec<typename LA_traits<T>::complextype> roots() const}{all roots as eigenvalues of the companion matrix (\code{gdiagonalize}, chapter~\ref{ch:nonclass}); for the floating point types
-	and \code{std::complex<double>} only (laerror for integer types).}
+	\code{double}, \code{float}, \code{std::complex<double>} and \code{std::complex<float>} only (laerror for integer types).}
-\lafunc{NRVec<T> realroots(const typename LA_traits<T>::normtype thr) const}{roots with $|\mathrm{Im}|<$ \code{thr}, real parts sorted ascending (\code{T = double}).}
+\lafunc{NRVec<T> realroots(const typename LA_traits<T>::normtype thr) const}{roots with $|\mathrm{Im}|<$ \code{thr}, real parts sorted ascending (floating point types).}
-\lafunc{T newton(const T x0, const typename LA_traits<T>::normtype thr=1e-14, const int maxit=1000) const}{Newton iteration from \code{x0} until $|p(x)|<$ \code{thr}; no convergence report.}
+\lafunc{T newton(const T x0, const typename LA_traits<T>::normtype thr=LA_precision<normtype>::newton_threshold(), const int maxit=1000) const}{Newton iteration from \code{x0} until $|p(x)|<$ \code{thr}; no convergence report.}
 \lafunc{template <typename T> Polynomial<T> polyfromroots(const NRVec<T> &roots, const int skip= -1)}{monic $\prod_{i\ne\mathrm{skip}}(x-r_i)$.}
 \lafunc{template <typename T> Polynomial<T> lagrange_interpolation(const NRVec<T> &x, const NRVec<T> &y)}{interpolating polynomial of degree $n-1$ through the $n$ points; laerror for repeated $x$.}
 \lafunc{template <typename T> NRMat<T> Sylvester(const Polynomial<T> &p, const Polynomial<T> &q)}{Sylvester matrix of dimension $\deg p+\deg q$ (its determinant is the resultant).}

doc/smat.tex

@@ -14,11 +14,10 @@ matrices) are returned as \code{NRMat<T>}.
 \code{NRSMat_from1<T>} (section~\ref{sec:smat:from1}) is the same class with the
 two-index \code{operator()} counting from 1.
 
-The file \code{smat.cc} contains explicit instantiations for the element types \code{double},
+The file \code{smat.cc} contains explicit instantiations for the element types \code{double}, \code{float},
-\code{std::complex<double>} and the integer types \code{char}, \code{short}, \code{int}, \code{long}, \code{long long} and
+\code{std::complex<double>}, \code{std::complex<float>} and the integer types \code{char}, \code{short}, \code{int}, \code{long}, \code{long long} and
-their unsigned variants (\code{float} is not instantiated).
+their unsigned variants.
-Most numerical members have BLAS-based specializations only for the two types \code{double} and
+Most numerical members have BLAS-based specializations only for the four floating point types. Some members are defined only for these types:
-\code{std::complex<double>}. Some members are defined only for these two types:
 \code{amax}, \code{amin}, \code{norm}, \code{dot}, \code{axpy}, \code{randomize}, \code{inverse}
 and the products with \code{NRMat} and \code{NRSMat}.
 
@@ -105,10 +104,10 @@ are CPU only (\code{NOT_GPU}).
 \lafunc{NRSMat(const T &a, const int n)}{Order $n$, all $n(n+1)/2$ stored elements (i.e.\ also the off-diagonal ones) set to \code{a}.}
 \lafunc{NRSMat(const T *a, const int n)}{Copy of $n(n+1)/2$ packed elements (lower triangle by rows) from \code{a}.}
 \lafunc{NRSMat(const NRSMat &rhs)}{Shallow copy.}
-\lafunc{NRSMat(const typename LA_traits_complex<T>::NRSMat_Noncomplex_type &rhs, bool imagpart = false)}{Complex matrix from a real one: \code{rhs} becomes the real part (\code{imagpart=false}) or the imaginary part (\code{imagpart=true}), the other part is zero. Defined only for \code{T=std::complex<double>}.}
+\lafunc{NRSMat(const typename LA_traits_complex<T>::NRSMat_Noncomplex_type &rhs, bool imagpart = false)}{Complex matrix from a real one: \code{rhs} becomes the real part (\code{imagpart=false}) or the imaginary part (\code{imagpart=true}), the other part is zero. Defined only for \code{T=std::complex<double>} and \code{std::complex<float>}.}
 \lafunc{explicit NRSMat(const NRMat<T> &rhs)}{Symmetric part $(A+A\Tr)/2$ of a square dense matrix (deep copy; nonsquare input is checked only in DEBUG).}
 \lafunc{explicit NRSMat(const NRVec<T> &rhs, const int n)}{Interprets a vector of length $n(n+1)/2$ as packed matrix of order $n$. \emph{Shares} the data with \code{rhs} (no copy); length checked only in DEBUG.}
-\lafunc{explicit NRSMat(const SparseMat<T> &rhs); explicit NRSMat(const SparseSMat<T> &rhs)}{Dense from sparse, see \ref{ch:sparse}.}
+\lafunc{explicit NRSMat(const SparseMat<T> &rhs); explicit NRSMat(const SparseSMat<T> &rhs); explicit NRSMat(const CSRMat<T> &rhs)}{Dense from sparse, see \ref{ch:sparse}; from \code{SparseSMat} and \code{CSRMat} the lower triangle is taken.}
 \lafunc{~NRSMat()}{Destructor, frees the data when the last reference disappears.}
 \lafunc{NRSMat & operator=(const NRSMat &rhs)}{Shallow copy, see section~\ref{sec:concepts:assign}.}
 \lafunc{NRSMat & operator|=(const NRSMat &rhs)}{Deep copy.}
@@ -161,20 +160,20 @@ generated from the in-place ones.
 \lafunc{void gemv(const T beta, NRVec<std::complex<T> > &r, const char trans, const T alpha, const NRVec<std::complex<T> > &x) const}{The same, real matrix and complex vectors.}
 \lafunc{void axpy(const T alpha, const NRSMat &x)}{$S \leftarrow S + \alpha x$. For hermitian complex matrices the result is hermitian only for real $\alpha$
 (not checked).}
-\lafunc{void rank2update(const T alpha, const NRVec<T> &x, const NRVec<T> &y)}{Rank-2 update $S \leftarrow S + \alpha x y\Hc + \bar\alpha y x\Hc$ (real: $\alpha(xy\Tr+yx\Tr)$); \code{dspr2}/\code{zhpr2} for \code{double}/\code{complex<double>}, generic loop otherwise. A symmetric rank-1 update $S \leftarrow S + \beta x x\Hc$ (real $\beta$) is obtained as \code{rank2update(beta/2,x,x)}; there is no separate rank-1 member.}
+\lafunc{void rank2update(const T alpha, const NRVec<T> &x, const NRVec<T> &y)}{Rank-2 update $S \leftarrow S + \alpha x y\Hc + \bar\alpha y x\Hc$ (real: $\alpha(xy\Tr+yx\Tr)$); \code{dspr2}/\code{zhpr2}/\code{sspr2}/\code{chpr2} for \code{double}/\code{complex<double>}/\code{float}/\code{complex<float>}, generic loop otherwise. A symmetric rank-1 update $S \leftarrow S + \beta x x\Hc$ (real $\beta$) is obtained as \code{rank2update(beta/2,x,x)}; there is no separate rank-1 member.}
 \lafunc{const T dot(const NRSMat &rhs) const}{$\sum_{i\ge j} \bar A_{ij} B_{ij}$, i.e.\ the dot product of the \emph{packed} vectors (off-diagonal elements counted once; \code{ddot}/\code{zdotc}).}
 \lafunc{const T dot(const NRVec<T> &rhs) const}{Dot product of the packed data with a vector of length $n(n+1)/2$.}
 \lafunc{const typename LA_traits<T>::normtype norm(const T scalar = (T)0) const}{Euclidean norm of the packed data of $S - \mathtt{scalar}\cdot 1$; off-diagonal elements are counted \emph{once}, so this is not the Frobenius norm of the full matrix.}
 \lafunc{const T trace() const}{Trace.}
 \lafunc{const T sum(bool onlytriangle=false) const}{Sum of all elements of the full matrix, or of the stored triangle only.}
 \lafunc{const NRVec<T> rsum() const; const NRVec<T> csum() const}{Row sums and column sums of the full matrix (equal for real, complex conjugate of each other for hermitian matrices).}
-\lafunc{inline const T amax() const; inline const T amin() const}{Stored element with the largest/smallest absolute value (complex: $|\Re|+|\Im|$); \code{double} and \code{complex<double>} only.}
+\lafunc{inline const T amax() const; inline const T amin() const}{Stored element with the largest/smallest absolute value (complex: $|\Re|+|\Im|$); floating point types only.}
 \lafunc{const T* diagonalof(NRVec<T> &r, const bool divide = 0, bool cache = false) const}{\code{divide=false}: store the diagonal into \code{r} and return \code{&r[0]}; \code{divide=true}: divide \code{r} elementwise by the nonzero diagonal elements and return \code{NULL}. \code{cache} is unused. Interface used by iterative solvers (\ref{ch:iterative}).}
-\lafunc{NRSMat inverse()}{Inverse via \code{NRMat<T>(*this).inverse()}, symmetrized back. \code{double} and \code{complex<double>} only.}
+\lafunc{NRSMat inverse()}{Inverse via \code{NRMat<T>(*this).inverse()}, symmetrized back. Floating point types only.}
 \lafunc{NRSMat& conjugateme(); const NRSMat conjugate() const}{Complex conjugation of all stored elements (no-op for real types).}
 \lafunc{NRSMat<typename LA_traits<T>::normtype> abs() const}{Elementwise absolute values.}
 \lafunc{NRSMat<typename LA_traits<T>::normtype> diffabs(const NRSMat &rhs) const}{Elementwise $|A_{ij}|-|B_{ij}|$.}
-\lafunc{void randomize(const typename LA_traits<T>::normtype &x)}{Uniformly distributed elements in $(-x,x)$; complex: real and imaginary parts independent, imaginary part of the diagonal set to zero. \code{double} and \code{complex<double>} only.}
+\lafunc{void randomize(const typename LA_traits<T>::normtype &x)}{Uniformly distributed elements in $(-x,x)$; complex: real and imaginary parts independent, imaginary part of the diagonal set to zero. Floating point types only.}
 \lafunc{const bool operator==(const NRSMat &rhs) const; const bool operator!=(const NRSMat &rhs) const}{Exact elementwise comparison.}
 \end{lareference}
 

doc/sparse.tex

@@ -1,8 +1,8 @@
 \chapter{Sparse matrices}\label{ch:sparse}
-\laheader{sparsemat.h, sparsemat.cc, sparsesmat.h, sparsesmat.cc}
+\laheader{sparsemat.h, sparsemat.cc, sparsesmat.h, sparsesmat.cc, csrmat.h, csrmat.cc}
 
 \section{Overview}\label{sec:sparse:overview}
-\LA{} offers two independent sparse matrix classes with different storage schemes, plus a small helper class:
+\LA{} offers three independent sparse matrix classes with different storage schemes, plus a small helper class:
 \begin{itemize}
 \item \code{SparseMat<T>} (section~\ref{sec:sparse:sparsemat}): an \emph{unordered linked list} of
   (row, column, value) triplets. Appending elements is cheap, duplicates are allowed and summed,
@@ -15,19 +15,21 @@
   exponentiating symmetric matrices (e.g.\ the unitary complex symmetric $\exp(\mathrm{i}H)$ with real symmetric $H$),
   but despite its name it is a general (also nonsymmetric and rectangular) sparse matrix; symmetry
   (hermiticity for complex types) is only a convention of some of its operations.
+\item \code{CSRMat<T>} (section~\ref{sec:sparse:csr}): the compressed sparse row format (row pointers, column
+  indices, values), read-efficient for products with vectors and matrices, e.g.\ in iterative solvers; built by
+  conversion from the other classes.
 \item \code{SemiSparseMat<T>} (section~\ref{sec:sparse:semisparse}): a dense diagonal plus a
   \code{SparseMat} off-diagonal part, just enough to be used in iterative eigensolvers.
 \end{itemize}
-\code{CSRMat<T>} (section~\ref{sec:sparse:csr}) is unfinished.
 
 Common definitions in \code{sparsemat.h}:
 \begin{lareference}
 \lafunc{typedef unsigned int SPMatindex; typedef int SPMatindexdiff}{Index type and index difference type. All row and column indices count from 0.}
-\lafunc{const double SPARSEEPSILON=1e-15}{Threshold: elements with absolute value below (or equal to) it are dropped by \code{simplify()} and by most constructors and accumulating operations.}
+\lafunc{const double SPARSEEPSILON=1e-15}{Kept for user code. The library itself uses the precision-dependent threshold \code{LA_precision<normtype>::sparse_epsilon()} (section~\ref{sec:concepts:traits}; $10^{-15}$ for double, i.e.\ $\epsilon_{\rm sp}$, and $10^{-7}$ for float): elements with absolute value below (or equal to) it are dropped by \code{simplify()} and by most constructors and accumulating operations; it is denoted $\epsilon_{\rm sp}$ below.}
 \lafunc{template<typename T> struct matel {T elem; SPMatindex row; SPMatindex col; matel *next;}}{List element of \code{SparseMat}; also the value type returned by the iterators of both classes.}
 \end{lareference}
-Both classes are explicitly instantiated for \code{double} and \code{std::complex<double>}.
+Both classes are explicitly instantiated for \code{double}, \code{float}, \code{std::complex<double>} and \code{std::complex<float>}.
-Several operations are implemented for \code{double} only. \code{SparseMat} nowhere conjugates when
+Several operations are implemented for the real types \code{double} and \code{float} only. \code{SparseMat} nowhere conjugates when
 it mirrors elements of a symmetric matrix (source comments: ``not OK for complex hermitean''), so
 its complex symmetric matrices are complex symmetric, not hermitian. \code{SparseSMat} stores both triangles
 and its mirroring operations follow the hermitian convention of \code{NRSMat} (section~\ref{sec:smat:hermitian}).
@@ -43,7 +45,7 @@ which no longer exists, is not included by any file and is not installed; it is
 The nonzero elements form a singly linked list of \code{matel<T>} (pointer \code{list}); new elements are
 prepended. The list is in no particular order and may contain several entries with the same
 $(i,j)$, which represent their sum; \code{simplify()} merges such duplicates, removes elements with
-$|a_{ij}|\le$ \code{SPARSEEPSILON} and leaves the list sorted by rows. Operations needing an ordering
+$|a_{ij}|\le$ $\epsilon_{\rm sp}$ and leaves the list sorted by rows. Operations needing an ordering
 (\code{simplify}, sparse products, \code{dot}) build private arrays of pointers sorted by rows or by columns
 (member \code{sort()}); these arrays and the cached element count \code{nonzero} are kept until the next
 operation which calls the private \code{unsort()}: \code{add()} (and everything built on it), \code{simplify},
@@ -86,8 +88,9 @@ Most modifying members call \code{copyonwrite()}: \code{simplify}, \code{operato
 \lafunc{SparseMat()}{Undefined empty matrix ($0\times0$, no reference count).}
 \lafunc{SparseMat(const SPMatindex n, const SPMatindex m)}{Defined empty (all zero) $n\times m$ matrix, not symmetric.}
 \lafunc{SparseMat(const SparseMat &rhs)}{Shallow copy; an empty matrix gets its own reference count.}
-\lafunc{explicit SparseMat(const NRMat<T> &rhs)}{From a dense matrix, elements with $|a_{ij}|>$ \code{SPARSEEPSILON}.}
+\lafunc{explicit SparseMat(const NRMat<T> &rhs)}{From a dense matrix, elements with $|a_{ij}|>$ $\epsilon_{\rm sp}$.}
 \lafunc{explicit SparseMat(const NRSMat<T> &rhs)}{From a packed symmetric matrix; symmetric flag set, lower triangle stored.}
+\lafunc{explicit SparseMat(const CSRMat<T> &rhs)}{From a \code{CSRMat} (section~\ref{sec:sparse:csr}): all stored elements, not symmetric.}
 \lafunc{~SparseMat()}{Destructor.}
 \lafunc{SparseMat & operator=(const SparseMat &rhs)}{Shallow copy (also copies the symmetric flag).}
 \lafunc{SparseMat & operator=(const T &a)}{Scalar matrix $a\cdot1$; the matrix must be square and dimensioned, old contents are discarded.}
@@ -107,8 +110,8 @@ duplicates summed), \code{explicit NRSMat<T>(const SparseMat<T>&)} (requires the
 \subsection{Element insertion and access}
 \begin{lareference}
 \lafunc{inline void add(const SPMatindex n, const SPMatindex m, const T elem)}{Prepend element $(n,m)$; it is summed with existing entries at the same position. No checks, no \code{copyonwrite()}, see section~\ref{sec:sparse:refcount}. The matrix must have been dimensioned (constructor with dimensions or \code{resize}).}
-\lafunc{void addsafe(const SPMatindex n, const SPMatindex m, const T elem)}{As \code{add}, but ignores $|$\code{elem}$|\le$ \code{SPARSEEPSILON}, calls \code{copyonwrite()} and creates the reference count of an undefined matrix (range check only with the lowercase \code{debug} macro).}
+\lafunc{void addsafe(const SPMatindex n, const SPMatindex m, const T elem)}{As \code{add}, but ignores $|$\code{elem}$|\le\epsilon_{\rm sp}$, calls \code{copyonwrite()} and creates the reference count of an undefined matrix (range check only with the lowercase \code{debug} macro).}
-\lafunc{void simplify(const double sparseepsilon=SPARSEEPSILON)}{Sum duplicates, remove elements with $|a_{ij}|\le$ \code{sparseepsilon}, sort by rows; symmetric matrices are first moved to the upper triangle ($i\le j$).}
+\lafunc{void simplify(const typename LA_traits<T>::normtype sparseepsilon=LA_precision<normtype>::sparse_epsilon())}{Sum duplicates, remove elements with $|a_{ij}|\le$ \code{sparseepsilon}, sort by rows; symmetric matrices are first moved to the upper triangle ($i\le j$).}
 \lafunc{unsigned int length() const}{Number of list elements (cached until the next modification, see section~\ref{sec:sparse:storage}).}
 \lafunc{unsigned int sort(int type) const}{Build the internal array sorted by rows (\code{type=0}) or columns (\code{type=1}) and return the number of elements. Logically const, but unfolds a symmetric matrix (\code{setunsymmetric}). Normally called internally only.}
 \lafunc{inline matel<T> *getlist() const; void setlist(matel<T> *l)}{Raw access to the list head (no reference count handling; \code{setlist} does not free the old list).}
@@ -132,7 +135,7 @@ Binary operators with a matrix operand must not combine a symmetric left operand
 \lafunc{const T trace() const}{Trace.}
 \lafunc{const typename LA_traits<T>::normtype norm(const T scalar=(T)0) const}{Frobenius norm of $A-\mathtt{scalar}\cdot1$ of the full matrix (symmetric off-diagonal elements counted twice). Calls \code{simplify()}.}
 \lafunc{const T dot(const SparseMat &rhs) const}{$\sum_{ij} a_{ij} b_{ij}$ without conjugation (also for complex types), general (not symmetric) matrices only; merge of the two lists sorted by rows.}
-\lafunc{const T dot(const NRMat<T> &rhs) const}{$\sum_{ij} a_{ij} b_{ij}$ with a dense matrix; \code{double} only.}
+\lafunc{const T dot(const NRMat<T> &rhs) const}{$\sum_{ij} a_{ij} b_{ij}$ with a dense matrix; \code{double} and \code{float} only.}
 \lafunc{const T* diagonalof(NRVec<T> &r, const bool divide=0, bool cache=false) const}{Square matrix: diagonal into \code{r} (\code{divide=false}, returns \code{&r[0]}) or divide \code{r} by the nonzero diagonal elements (\code{divide=true}, returns \code{NULL}). Rectangular matrix: the diagonal of $A\Tr A$ (assuming a simplified matrix), for preconditioning. \code{cache} is unused.}
 \end{lareference}
 
@@ -140,7 +143,7 @@ Binary operators with a matrix operand must not combine a symmetric left operand
 \begin{lareference}
 \lafunc{inline const NRVec<T> operator*(const NRVec<T> &rhs) const}{$A x$.}
 \lafunc{void gemv(const T beta, NRVec<T> &r, const char trans, const T alpha, const NRVec<T> &x, bool treat_as_symmetric=false) const}{$r \leftarrow \beta r + \alpha\,\mathrm{op}(A)x$; forwards to \code{NRVec<T>::gemv}. Any \code{trans} other than \code{'n'} means transposition (no conjugation). With \code{treat_as_symmetric} a general matrix is used as if it had the symmetric flag. The sizes of \code{x} and (in DEBUG) of the result are checked.}
-\lafunc{inline const NRMat<T> operator*(const NRMat<T> &rhs) const}{$A B$ via \code{NRMat<T>::gemm} with a sparse first operand; \code{double} only (\code{complex<double>} raises ``not implemented yet'').}
+\lafunc{inline const NRMat<T> operator*(const NRMat<T> &rhs) const}{$A B$ via \code{NRMat<T>::gemm} with a sparse first operand; \code{double} and \code{float} only (the complex types raise ``not implemented yet'').}
 \lafunc{const SparseMat operator*(const SparseMat &rhs) const}{Sparse product of two general matrices (error if one is symmetric); the result is simplified.}
 \lafunc{void gemm(const T beta, const SparseMat &a, const char transa, const SparseMat &b, const char transb, const T alpha)}{$C \leftarrow \beta C + \alpha\,\mathrm{op}(A)\,\mathrm{op}(B)$ for general \code{a}, \code{b}; if \code{*this} is symmetric (possible only with $\beta\neq0$) only elements with $i\le j$ are added. Result simplified. \code{a} and \code{b} must not be \code{*this} (checked in DEBUG).}
 \end{lareference}
@@ -148,7 +151,7 @@ Related functions in other classes:
 \code{NRVec<T>::gemv(const T beta, const SparseMat<T> &a, const char trans, const T alpha, const NRVec &x, const bool treat_as_symmetric=false)},
 \code{NRVec<T>::operator*(const SparseMat<T>&)} ($x\Tr A$),
 \code{NRMat<T>::gemm(const T &beta, const SparseMat<T> &a, const char transa, const NRMat &b, const char transb, const T &alpha)}
-(\code{double} only) and \code{NRMat<T>::operator+=}, \code{operator-=(const SparseMat<T>&)}.
+(\code{double} and \code{float} only) and \code{NRMat<T>::operator+=}, \code{operator-=(const SparseMat<T>&)}.
 
 \subsection{Transposition, permutations, direct sums}
 \begin{lareference}
@@ -180,7 +183,7 @@ e.g.\ \code{NRVec<SparseMat<T>>} can be written and read; the ORCA format is def
 \lafunc{inline const SparseMat<T> commutator(const SparseMat<T> &x, const SparseMat<T> &y, const bool trx=0, const bool tryy=0)}{$[\mathrm{op}(x),\mathrm{op}(y)] = xy-yx$, via two \code{gemm} calls (general matrices only).}
 \lafunc{inline const SparseMat<T> anticommutator(const SparseMat<T> &x, const SparseMat<T> &y, const bool trx=0, const bool tryy=0)}{$xy+yx$.}
 \end{lareference}
-In \ref{ch:nonclass}: \code{realpart()} and \code{imagpart()} of a \code{SparseMat<complex<double>>}. The generic
+In \ref{ch:nonclass}: \code{realpart()} and \code{imagpart()} of a \code{SparseMat<complex<double>>} or \code{SparseMat<complex<float>>}. The generic
 matrix functions in \code{matexp.h} (e.g.\ \code{exp()}) work with \code{SparseMat}, and the iterative solvers
 of \ref{ch:iterative} use its \code{gemv}, \code{diagonalof}, \code{issymmetric}, \code{nrows} interface.
 
@@ -245,10 +248,10 @@ copies).
 \lafunc{SparseSMat()}{Undefined empty matrix.}
 \lafunc{explicit SparseSMat(const SPMatindex n); explicit SparseSMat(const SPMatindex n, const SPMatindex m)}{Empty (zero) $n\times n$ or $n\times m$ matrix.}
 \lafunc{SparseSMat(const SparseSMat &rhs)}{Shallow copy.}
-\lafunc{explicit SparseSMat(const NRSMat<T> &rhs)}{From a packed symmetric (complex: hermitian) matrix (both triangles inserted, the upper one conjugated, $|a_{ij}|>$ \code{SPARSEEPSILON}).}
+\lafunc{explicit SparseSMat(const NRSMat<T> &rhs)}{From a packed symmetric (complex: hermitian) matrix (both triangles inserted, the upper one conjugated, $|a_{ij}|>$ $\epsilon_{\rm sp}$).}
 \lafunc{explicit SparseSMat(const NRMat<T> &rhs)}{From a dense $n\times m$ matrix, nonzero elements copied as they are (no symmetrization).}
 \lafunc{explicit SparseSMat(const SparseMat<T> &rhs)}{From a \code{SparseMat} (same dimensions); duplicates are summed, a symmetric \code{SparseMat} is unfolded without conjugation (complex symmetric, as \code{SparseMat} defines it).}
-\lafunc{explicit SparseSMat(const CSRMat<T> &rhs)}{Declared but not implemented (\code{CSRMat} is not supported).}
+\lafunc{explicit SparseSMat(const CSRMat<T> &rhs)}{From a \code{CSRMat} (section~\ref{sec:sparse:csr}): all stored elements, without mirroring.}
 \lafunc{~SparseSMat()}{Destructor.}
 \lafunc{SparseSMat & operator=(const SparseSMat &rhs)}{Shallow copy.}
 \lafunc{SparseSMat & operator=(const T &a)}{Scalar matrix $a\cdot1$ (diagonal elements are created even for $a=0$).}
@@ -266,7 +269,7 @@ Dense from sparse: \code{explicit NRMat<T>(const SparseSMat<T>&)} copies all sto
 \lafunc{inline void add(const SPMatindex n, const SPMatindex m, const T elem, const bool both=true)}{Add \code{elem} to element $(n,m)$ and, if \code{both} and $n\ne m$, its complex conjugate to $(m,n)$. Creates the row map if needed; no \code{copyonwrite()}; range check in DEBUG only. Resulting zeros remain until \code{simplify()}.}
 \lafunc{inline const std::map<SPMatindex,T> *line(const SPMatindex n) const}{Read-only access to row $n$ (\code{NULL} if empty).}
 \lafunc{inline ROWTYPE & operator[](const SPMatindex i)}{Row $i$ as a reference to the pointer \code{std::map<SPMatindex,T>*} (\code{typedef ROWTYPE}), possibly \code{NULL}. Non-const only.}
-\lafunc{unsigned long long simplify()}{Remove elements with $|a_{ij}|<$ \code{SPARSEEPSILON} and empty rows; returns the number of remaining elements.}
+\lafunc{unsigned long long simplify()}{Remove elements with $|a_{ij}|<$ $\epsilon_{\rm sp}$ and empty rows; returns the number of remaining elements.}
 \lafunc{inline unsigned long long length()}{\code{simplify()} (non-const).}
 \lafunc{int nrows() const; int ncols() const; inline bool issymmetric() const}{Dimensions; \code{issymmetric()} is always true.}
 \lafunc{iterator begin() const; iterator end() const}{Iterator over all stored elements, row by row in increasing column order. \code{*it} is a \code{matel<T>} \emph{copy} held in the iterator (writing to it does not change the matrix). It is meant for comparison with \code{end()} or via \code{it.notend()}; \code{iterator(const SparseSMat&)} starts at the first element.}
@@ -288,7 +291,7 @@ Dense from sparse: \code{explicit NRMat<T>(const SparseSMat<T>&)} copies all sto
 \lafunc{const T* diagonalof(NRVec<T> &r, const bool divide=0, bool cache=false) const}{As for \code{SparseMat} (square matrices only).}
 \lafunc{SparseSMat transpose(bool conj=false) const}{Transposed (optionally conjugated) copy.}
 \lafunc{void transposeme() const}{Raises an error (not implemented).}
-\lafunc{SparseSMat<T> cholesky(void) const}{Pivoted Cholesky decomposition of a positive semidefinite symmetric/hermitian matrix (not checked): returns $R$ with $R\Hc R = A$ (\code{result.transpose(true)*result} reproduces the input). The pivoting is done a priori by sorting the diagonal; pivots with real part below \code{CHOLESKYEPSILON}$=10^{-16}$ are skipped (rank-deficient case), elements below \code{SPARSEEPSILON} are dropped. $R$ is upper triangular only in the permuted basis.}
+\lafunc{SparseSMat<T> cholesky(void) const}{Pivoted Cholesky decomposition of a positive semidefinite symmetric/hermitian matrix (not checked): returns $R$ with $R\Hc R = A$ (\code{result.transpose(true)*result} reproduces the input). The pivoting is done a priori by sorting the diagonal; pivots with real part below \code{LA_precision<normtype>::cholesky_epsilon()} ($10^{-16}$ for double, the old \code{CHOLESKYEPSILON}, $10^{-8}$ for float) are skipped (rank-deficient case), elements below $\epsilon_{\rm sp}$ are dropped. $R$ is upper triangular only in the permuted basis.}
 \lafunc{SparseSMat submatrix(const int fromrow, const int torow, const int fromcol, const int tocol) const}{Block (inclusive bounds).}
 \lafunc{void storesubmatrix(const int fromrow, const int fromcol, const SparseSMat &rhs)}{Store \code{rhs} into the block starting at $(\mathtt{fromrow},\mathtt{fromcol})$: the old elements of the block are removed first (as \code{NRMat::storesubmatrix}); calls \code{copyonwrite()}.}
 \end{lareference}
@@ -302,19 +305,108 @@ Dense from sparse: \code{explicit NRMat<T>(const SparseSMat<T>&)} copies all sto
 \lafunc{SparseSMat<T> otimes_sparse(const NRVec<T> &lhs, const NRVec<T> &rhs, const bool conjugate=false, const T &scale=1)}{\emph{Symmetrized} outer product, symmetric (complex: hermitian) regardless of the input vectors (unlike \code{SparseMat}):
 with $M=\mathtt{scale}\cdot l\,r\Tr$ (or $l\,r\Hc$ if \code{conjugate}) the result is $S=M+M\Hc$ ($=l\,r\Tr+r\,l\Tr$ for real types),
 including the doubled diagonal $S_{ii}=M_{ii}+M_{ii}^*$. Note that \code{otimes_sparse(x,x,true)} is $2\,x x\Hc$; use \code{scale=0.5} for $x x\Hc$.
-Elements with $|S_{ij}|\le$ \code{SPARSEEPSILON} are omitted. Both vectors must have the same length (\code{laerror} otherwise).}
+Elements with $|S_{ij}|\le$ $\epsilon_{\rm sp}$ are omitted. Both vectors must have the same length (\code{laerror} otherwise).}
 \lafunc{SparseSMat<T> NRMat<T>::operator*(const SparseSMat<T> &rhs) const}{Declared in mat.h: dense times sparse, general product $AS$ with a sparse result (row $i$ accumulated as $\sum_k A_{ik}\,(\text{row }k\text{ of }S)$, zero $A_{ik}$ skipped; simplified). Instantiated for \code{double} and \code{std::complex<double>}.}
 \end{lareference}
 In \ref{ch:nonclass}: \code{complexmatrix()}, \code{realmatrix()} and \code{imagmatrix()} build a
-\code{SparseSMat<complex<double>>} from real ones. The generic \code{exp()} of \code{matexp.h} works with
+\code{SparseSMat<complex<double>>} (\code{SparseSMat<complex<float>>}) from real ones. The generic \code{exp()} of \code{matexp.h} works with
 \code{SparseSMat}.
 
 \section{CSRMat}\label{sec:sparse:csr}
-\laheader{csrmat.h}
+\laheader{csrmat.h, csrmat.cc}
-\code{CSRMat<T>} is meant to hold a matrix in compressed sparse row format (row pointers, column indices,
+\code{CSRMat<T>} stores a matrix in the compressed sparse row format: the elements of row $i$ are \code{v[k]} in
-values, optionally on the GPU via cuSPARSE) for fast products. The class is unfinished: only the declarations
+column \code{col[k]} for $k=$ \code{rowptr[i]}$,\ldots,$\code{rowptr[i+1]}$-1$, with the columns strictly ascending within
-exist, \code{csrmat.cc} contains no implementation and no instantiation. It is \emph{not supported} and must not be used;
+each row (an invariant of the class). The format is read-efficient: $Ax$ is a row-wise dot product without any
-the declarations (also \code{NRMat(const CSRMat&)} and \code{SparseSMat(const CSRMat&)}) are kept for possible future work.
+indirection in the result, which makes it the fastest of the sparse classes in iterative solvers (\ref{ch:iterative})
+and matrix functions with a fixed sparsity pattern. It cannot be built element by element; a matrix is assembled as a
+\code{SparseMat} (\code{add}) or a \code{SparseSMat} (or as three arrays) and converted, in $O(n_{\rm nz}+n+m)$.
+
+The full matrix is always stored, also for symmetric and hermitian matrices. The flag \code{symmetric} only
+\emph{declares} the matrix hermitian, $A=A^\dagger$ (real symmetric for real types); it is not checked and never changes the
+arithmetic, but it is what \code{issymmetric()} returns to the eigensolvers (\code{davidson} uses the hermitian algorithm,
+\code{lanczos} requires it). This is the convention of \code{NRSMat} and \code{SparseSMat}, not the complex symmetric
+one of \code{SparseMat}.
+
+The three arrays are \code{NRVec} members (\code{rowptr} of length $n+1$, \code{col} and \code{v} of length $n_{\rm nz}$, the
+number of stored elements); each of them shares its data independently (section~\ref{sec:concepts:refcount}). Operations which
+change only the values keep the index arrays shared, e.g.\ \code{B=A*2.}, \code{A+=a} with all diagonal elements stored, or
+\code{realmatrix(A)}; \code{copyonwrite()} copies all three. Conversions store only elements with $|a_{ij}|>\epsilon_{\rm sp}$,
+whereas arithmetic keeps elements which cancel to zero, until \code{simplify()} is called. $n_{\rm nz}$ is limited to $2^{31}-1$
+(the \code{NRVec} length). The class is instantiated for \code{double}, \code{float}, \code{std::complex<double>} and
+\code{std::complex<float>}; all operations are CPU only (\code{laerror} for vectors and matrices on the GPU), the device-dependent
+format is the one of cuSPARSE, so a GPU version can be added later.
+
+\subsection{Construction, conversion, access}
+\begin{lareference}
+\lafunc{CSRMat()}{Empty $0\times0$ matrix.}
+\lafunc{explicit CSRMat(const SPMatindex n, const SPMatindex m); explicit CSRMat(const SPMatindex n)}{Zero $n\times m$ resp.\ $n\times n$ matrix.}
+\lafunc{CSRMat(const SPMatindex n, const SPMatindex m, const NRVec<SPMatindex> &rowptr, const NRVec<SPMatindex> &col, const NRVec<T> &v)}{From the three arrays, which are shared (not copied). In DEBUG their consistency, the column range and the ascending columns are checked.}
+\lafunc{explicit CSRMat(const SparseMat<T> &rhs)}{Duplicates are summed, elements with $|a_{ij}|\le\epsilon_{\rm sp}$ dropped. A symmetric \code{SparseMat} is unfolded without conjugation (complex symmetric, as \code{SparseMat} defines it); the flag is set for the real types only.}
+\lafunc{explicit CSRMat(const SparseSMat<T> &rhs); explicit CSRMat(const NRMat<T> &rhs)}{Elements with $|a_{ij}|>\epsilon_{\rm sp}$; the flag is not set (call \code{setsymmetric()} if appropriate).}
+\lafunc{explicit CSRMat(const NRSMat<T> &rhs)}{Hermitian as \code{NRSMat}: the upper triangle is the conjugate of the stored lower one; the flag is set.}
+\lafunc{void copyonwrite(bool detachonly=false, bool deep=true); void resize(const SPMatindex n, const SPMatindex m); void dealloc(); void clear(); void swap(CSRMat &rhs)}{As for the other classes; \code{resize} gives a zero matrix and clears the flag, \code{clear} keeps it.}
+\lafunc{SPMatindex nrows() const; SPMatindex ncols() const; int length() const}{Dimensions and $n_{\rm nz}$.}
+\lafunc{bool issymmetric() const; void setsymmetric(); void setunsymmetric()}{The hermitian flag; \code{setsymmetric} only declares (square matrices only).}
+\lafunc{const NRVec<SPMatindex> & getrowptr() const; const NRVec<SPMatindex> & getcol() const; const NRVec<T> & getval() const}{Read access to the arrays.}
+\lafunc{const T operator()(const SPMatindex i, const SPMatindex j) const}{Element $a_{ij}$ by binary search in row $i$, zero if not stored. There is no write access to single elements.}
+\lafunc{iterator begin() const; iterator end() const}{Read-only iterator over the stored elements in row-major order: \code{p->row}, \code{p->col}, \code{p->elem} (a \code{matel<T>}), \code{p.notend()}.}
+\end{lareference}
+Other classes from \code{CSRMat}: \code{explicit NRMat<T>(const CSRMat<T>&)} (full matrix), \code{explicit NRSMat<T>(const CSRMat<T>&)}
+(lower triangle, square only, as from \code{SparseSMat}), \code{explicit SparseMat<T>(const CSRMat<T>&)} (not symmetric),
+\code{explicit SparseSMat<T>(const CSRMat<T>&)} and \code{explicit NRVec<T>(const CSRMat<T>&)} (for a $1\times m$ or $n\times 1$ matrix).
+
+\subsection{Arithmetic}
+\begin{lareference}
+\lafunc{CSRMat & operator*=(const T &a); const CSRMat operator*(const T &a) const}{Scaling of the values (BLAS); the index arrays stay shared, $a=0$ gives the zero matrix.}
+\lafunc{CSRMat & operator=(const T &a); void identity()}{$a\cdot1$ resp.\ the unit matrix (square only).}
+\lafunc{CSRMat & operator+=(const T &a); CSRMat & operator-=(const T &a); const CSRMat operator+(const T &a) const; const CSRMat operator-(const T &a) const}{$A\pm a\cdot1$ (square only): in place if all diagonal elements are stored, otherwise by a merge.}
+\lafunc{void axpy(const T alpha, const CSRMat &x, const bool transp=false)}{$A\leftarrow A+\alpha x$ (or $\alpha x\Tr$): merge of the sorted rows in two passes; for the same structure (shared or equal index arrays) a BLAS \code{axpy} of the values. Elements which cancel stay stored.}
+\lafunc{CSRMat & operator+=(const CSRMat &rhs); CSRMat & operator-=(const CSRMat &rhs); const CSRMat operator+(const CSRMat &rhs) const; const CSRMat operator-(const CSRMat &rhs) const}{Via \code{axpy}.}
+\lafunc{const typename LA_traits<T>::normtype norm(const T scalar=(T)0) const}{Frobenius norm of $A-\mathrm{scalar}\cdot1$.}
+\lafunc{const T dot(const CSRMat &rhs) const; const T dot(const NRMat<T> &rhs) const}{$\sum_{ij}\bar a_{ij}b_{ij}$, conjugating the left operand as \code{NRMat::dot} (unlike \code{SparseMat::dot}).}
+\lafunc{const T trace() const}{Sum of the diagonal elements.}
+\lafunc{void simplify(const typename LA_traits<T>::normtype sparseepsilon=LA_precision<normtype>::sparse_epsilon())}{Removes the elements with $|a_{ij}|\le$ \code{sparseepsilon}.}
+\end{lareference}
+The result of an addition is declared hermitian if both operands are (a matrix without stored elements counts as hermitian) and the
+scalar factor is real; multiplying by or adding a non-real scalar clears the flag. In other classes:
+\code{NRMat<T>::operator+=(const CSRMat<T>&)} and \code{operator-=}.
+
+\subsection{Products}
+\begin{lareference}
+\lafunc{void gemv(const T beta, NRVec<T> &r, const char trans, const T alpha, const NRVec<T> &x) const}{$r\leftarrow\beta r+\alpha\,\mathrm{op}(A)x$, \code{trans} \code{'n'}, \code{'t'} or \code{'c'} (conjugate transposition, the same as \code{'t'} for real types); forwards to \code{NRVec<T>::gemv}. The size of \code{x} and (in DEBUG) of the result are checked, $\beta=0$ clears the result.}
+\lafunc{inline const NRVec<T> operator*(const NRVec<T> &rhs) const}{$Ax$.}
+\lafunc{const T* diagonalof(NRVec<T> &r, const bool divide=0, bool cache=false) const}{Diagonal (binary search in each row), or $r$ divided by it elementwise (zero diagonal elements skipped). For a non-square matrix the diagonal of $A\Tr A$ ($\sum_i a_{ij}^2$, no conjugation, as \code{SparseMat}), used as the preconditioner of \code{conjgrad}. \code{cache} is ignored.}
+\lafunc{inline const NRMat<T> operator*(const NRMat<T> &rhs) const}{$AB$ with a dense result, via \code{NRMat<T>::gemm} below.}
+\lafunc{void gemm(const T beta, const CSRMat &a, const char transa, const CSRMat &b, const char transb, const T alpha)}{$C\leftarrow\beta C+\alpha\,\mathrm{op}(A)\,\mathrm{op}(B)$ with a sparse result (\code{trans} as for \code{gemv}), by the row-wise algorithm of Gustavson with a dense accumulator \cite{sparse:gustavson1978} in two passes; the rows of the result are sorted, elements which cancel stay stored, the result is not declared hermitian. With $\beta=0$ the dimensions of \code{*this} do not matter.}
+\lafunc{inline const CSRMat operator*(const CSRMat &rhs) const}{$AB$.}
+\end{lareference}
+In other classes: \code{NRVec<T>::gemv(const T beta, const CSRMat<T> &a, const char trans, const T alpha, const NRVec &x)},
+\code{NRVec<T>::operator*(const CSRMat<T>&)} ($x\Tr A$),
+\code{NRMat<T>::gemm(const T &beta, const CSRMat<T> &a, const char transa, const NRMat &b, const char transb, const T &alpha)}
+(all combinations of \code{'n'}, \code{'t'}, \code{'c'}) and \code{NRMat<T>::operator*(const CSRMat<T>&)} (dense times sparse, dense result).
+The sparse kernels are plain loops (there is no sparse BLAS in the libraries used).
+
+\subsection{Transposition, permutations, direct sums and products}
+\begin{lareference}
+\lafunc{const CSRMat transpose(bool conj=false) const; void transposeme(bool conj=false)}{$A\Tr$ or $A\Hc$ by a counting sort, $O(n_{\rm nz}+n+m)$; the flag is kept.}
+\lafunc{void permuteme_rows(const NRPerm<int> &p, const bool inverse=false); void permuteme_cols(const NRPerm<int> &p, const bool inverse=false); void permuteme_both(const NRPerm<int> &p, const NRPerm<int> &q, const bool inverse=false)}{The convention of \code{SparseMat}: element $(i,j)$ moves to row $p_{i+1}-1$ resp.\ column $q_{j+1}-1$ (matrix indices from 0, permutations from 1, optionally inverted). Since the full matrix is stored, \code{permuteme_rows} permutes only the rows also of a hermitian matrix (unlike a symmetric \code{SparseMat}); the flag is cleared, except by \code{permuteme_both} with $p=q$.}
+\lafunc{const CSRMat permuted_rows(...) const; const CSRMat permuted_cols(...) const; const CSRMat permuted_both(...) const}{Out-of-place versions.}
+\lafunc{void permuterows(const NRVec<SPMatindex> &p); void permutecolumns(const NRVec<SPMatindex> &p); void permuteindices(const NRVec<SPMatindex> &p)}{As for \code{SparseMat}: $i\to p_i$ (indices from 0), \code{permuteindices} for rows and columns.}
+\lafunc{CSRMat & oplusequal(const CSRMat &rhs); const CSRMat oplus(const CSRMat &rhs) const}{Direct sum $A\oplus B$; also with \code{NRMat} and \code{NRSMat} (converted as above, i.e.\ \code{NRSMat} as hermitian).}
+\lafunc{const CSRMat otimes(const CSRMat &rhs) const}{Direct (Kronecker) product, $(A\otimes B)_{i n_B+k,\,j m_B+l}=a_{ij}b_{kl}$ as \code{NRMat::operator|}; also with \code{NRMat} and \code{NRSMat}.}
+\end{lareference}
+The results of \code{oplus} and \code{otimes} are declared hermitian if both operands are.
+
+\subsection{Input/output and free functions}
+\begin{lareference}
+\lafunc{void put(int fd, bool dimensions=1, bool transp=0, bool orcaformat=false) const; void get(int fd, bool dimensions=1, bool transp=0, bool orcaformat=false)}{Binary I/O: if \code{dimensions}, $n$ and $m$ (\code{SPMatindex}); the flag (\code{int}); the three arrays by \code{NRVec::put} with their lengths. \code{transp} writes or reads the transposed matrix; \code{orcaformat=true} raises \code{laerror}.}
+\lafunc{std::ostream& operator<<(std::ostream &s, const CSRMat<T> &x); std::istream& operator>>(std::istream &s, CSRMat<T> &x)}{Text I/O in the format of \code{SparseMat} (read through a \code{SparseMat}, so duplicates are summed); the flag is not written.}
+\lafunc{inline const CSRMat<T> commutator(const CSRMat<T> &x, const CSRMat<T> &y, const bool trx=0, const bool tryy=0); inline const CSRMat<T> anticommutator(...)}{$xy\mp yx$ via two \code{gemm} calls.}
+\end{lareference}
+In \ref{ch:nonclass}: \code{realpart()}, \code{imagpart()} of a complex \code{CSRMat} and \code{realmatrix()}, \code{imagmatrix()}, \code{complexmatrix()}
+of real ones; only the values are converted (BLAS), the index arrays stay shared unless elements which became zero are removed.
+The real part of a hermitian matrix and \code{realmatrix} of a declared symmetric one keep the flag. The templates of \code{matexp.h}
+(\code{exp()}, \code{sincos()}, \code{exptimes()}) and the iterative solvers of \ref{ch:iterative} work with \code{CSRMat}.
 
 \section{Examples}\label{sec:sparse:examples}
 \begin{lstlisting}
@@ -353,6 +445,15 @@ h.add(2,2,1.);
 SparseSMat<double> h2 = h*h;             //result stays symmetric
 NRVec<double> w = h*x;
 std::cout << y << z << w << NRMat<double>(h2) << sd;
+
+//compressed row format: assembled as SparseMat, then converted
+SparseMat<double> hb(4,4);
+hb.add(0,0,1.); hb.add(1,0,0.2); hb.add(0,1,0.2); hb.add(3,3,2.);
+CSRMat<double> hc(hb);
+hc.setsymmetric();                       //declare it symmetric, e.g. for davidson
+NRVec<double> u = hc*x;
+CSRMat<std::complex<double> > ev = exp(imagmatrix(hc)); //exp(iH), sparse
+CSRMat<double> hc2 = hc*2.;              //shares rowptr and col with hc
 }
 \end{lstlisting}
 
@@ -364,4 +465,7 @@ std::cout << y << z << w << NRMat<double>(h2) << sd;
 \item In a symmetric \code{SparseMat} never store both $a_{ij}$ and $a_{ji}$; \code{setsymmetric()} does not check this.
 \item \code{SparseSMat::gemm} computes $A\Tr B$ (or $A\Hc B$); use \code{operator*} for the general product.
 \item Complex \code{SparseMat} matrices are complex symmetric, never hermitian, in all mirroring operations, whereas \code{SparseSMat} (\code{add} with \code{both=true}, conversion from \code{NRSMat}, \code{otimes_sparse}) is hermitian.
+\item \code{CSRMat} cannot be filled element by element; assemble a \code{SparseMat} or \code{SparseSMat} and convert it.
+\item The flag of \code{CSRMat} only declares hermiticity. Conversions from \code{NRMat}, \code{SparseSMat} and arrays (and from a complex
+  symmetric \code{SparseMat}) do not set it; call \code{setsymmetric()} for a hermitian matrix before \code{davidson} or \code{lanczos}.
 \end{itemize}

doc/tensor.tex

@@ -18,8 +18,8 @@ Typical uses are many-body quantities (amplitudes, integrals, density matrices)
 in groups of indices should be exploited for storage.
 Limitations stated in the source:
 \begin{itemize}
-\item the implementation is ``rudimentary''; only \code{Tensor<double>} and \code{Tensor<std::complex<double> >}
+\item the implementation is ``rudimentary''; only \code{Tensor<double>}, \code{Tensor<std::complex<double> >} and their single precision
-	are instantiated (tensor.cc);
+	counterparts \code{Tensor<float>}, \code{Tensor<std::complex<float> >} are instantiated (tensor.cc);
 \item index types \code{LA_index} and \code{LA_largeindex} are \code{int}, so the total size is limited
 	to $2^{31}$ elements (\code{NRVec} limitation);
 \item symmetry \emph{between} index groups (e.g.\ $(ij|kl)=(kl|ij)$) is not supported; one would have to
@@ -291,7 +291,7 @@ Names are permuted together with the indices unless stated otherwise.
 \begin{lareference}
 \lafunc{void apply_permutation_algebra(const Tensor &rhs, const PermutationAlgebra<int,T> &pa, bool inverse=false, T alpha=1, T beta=0, bool conjugate_by_parity=false)}{$\mathtt{*this} \leftarrow \beta\,\mathtt{*this}$, then for all stored elements $I$ of \code{*this}: $\mathtt{this}(I) \mathrel{+}= \alpha \sum_P c_P\, \mathtt{rhs}(P(I))$, with $P$ acting on the flat index list (inverse permutations if \code{inverse}); with \code{conjugate_by_parity} odd permutations take the complex conjugate. Can represent general (anti)symmetrizers, e.g.\ \code{general_antisymmetrizer} (chapter~\ref{ch:permutation}). \code{*this} may have a symmetry which \code{rhs} lacks. The names of \code{*this} are taken from (or checked against) the names of \code{rhs} permuted by the first permutation. Element-by-element, not optimally efficient.}
 \lafunc{void apply_permutation_algebra(const NRVec<Tensor> &rhsvec, const PermutationAlgebra<int,T> &pa, bool inverse=false, T alpha=1, T beta=0)}{the same with \code{rhs} being the outer product of the tensors in \code{rhsvec}, which is not formed explicitly (e.g.\ Grassmann products building a density matrix from cumulants); rather inefficient.}
-\lafunc{NRVec<NRMat<T> > Tucker(typename LA_traits<T>::normtype thr=1e-12, bool inverseorder=false)}{HOSVD/Tucker decomposition: \code{*this} is replaced by the flat core tensor and the factor matrices (one per index, columns orthonormal) are returned; singular values below \code{thr} are dropped, which reduces the corresponding range of the core.}
+\lafunc{NRVec<NRMat<T> > Tucker(typename LA_traits<T>::normtype thr=LA_precision<normtype>::svd_threshold(), bool inverseorder=false)}{HOSVD/Tucker decomposition: \code{*this} is replaced by the flat core tensor and the factor matrices (one per index, columns orthonormal) are returned; singular values below \code{thr} are dropped, which reduces the corresponding range of the core. The tensor is $X=G\times_1U_1\cdots\times_rU_r$ with the core $G$ and the factor matrices $U_k$, contracted without complex conjugation ($G=X\times_1U_1^\dagger\cdots\times_rU_r^\dagger$).}
 \lafunc{Tensor inverseTucker(const NRVec<NRMat<T> > &x, bool inverseorder=false) const}{rebuild the tensor from the core (\code{*this}) and the matrices returned by \code{Tucker} with the same \code{inverseorder}.}
 \lafunc{Tensor linear_transform(const NRVec<NRMat<T> > &x) const}{transform every index of group $g$ by the matrix \code{x[g]} (\code{x[g].ncols()} must equal the range; the new range is \code{x[g].nrows()}), preserving the group symmetries: $t'_{\dots a \dots} = \sum_i x_{ai}\, t_{\dots i\dots}$.}
 \lafunc{Tensor linear_transform(const int g, const Tensor<T> &x) const; Tensor linear_transform(const int g, const NRMat<T> &x) const}{transform only group \code{g}, keeping its position and symmetry; \code{x} must be a flat rank-2 tensor (useful for raising/lowering indices) or a matrix.}

doc/vec.tex

@@ -66,6 +66,7 @@ one: \code{rhs} becomes the real part (or the imaginary part if \code{imagpart})
 $n(n+1)/2$ elements. \emph{Shares} the data with \code{S} (no copy).}
 \lafunc{explicit NRVec(const NRMat<T> &rhs)}{View of the row-major storage of an $n\times m$ matrix as a vector of $nm$ elements.
 \emph{Shares} the data with \code{rhs}; if the library is compiled with \code{MATPTR}, the data are copied instead.}
+\lafunc{explicit NRVec(const CSRMat<T> &rhs)}{The same from a compressed row sparse matrix (section~\ref{sec:sparse:csr}).}
 \lafunc{explicit NRVec(const SparseMat<T> &rhs)}{Dense vector from a sparse matrix having one row or one column (otherwise
 \code{laerror}); duplicate entries are summed. Defined in \texttt{sparsemat.cc} for \code{double} and \code{std::complex<double>}.}
 \lafunc{~NRVec()}{Decrements the reference count, frees the data when it reaches zero.}
@@ -162,6 +163,7 @@ with $y=$ \code{*this}; \code{trans} is \code{'n'} ($A$), \code{'t'} ($A\Tr$) or
 \code{*this} must already have the correct size (checked in DEBUG builds) and defined contents if $\beta\neq0$.}
 \lafunc{void gemv(const T beta, const NRSMat<T> &a, const char trans, const T alpha, const NRVec &x)}{Same for a packed
 symmetric (real) or Hermitian (complex) matrix (BLAS spmv/hpmv); \code{trans}: for complex \code{'t'} gives $a\Tr x = a^* x$, \code{'n'} and \code{'c'} give $a x$; ignored for real matrices.}
+\lafunc{void gemv(const T beta, const CSRMat<T> &a, const char trans, const T alpha, const NRVec &x)}{The same for a compressed row sparse matrix, \code{trans} \code{'n'}, \code{'t'} or \code{'c'} (section~\ref{sec:sparse:csr}).}
 \lafunc{void gemv(const T beta, const SparseMat<T> &a, const char trans, const T alpha, const NRVec &x, const bool treat_as_symmetric = false)}{Same
 for a sparse matrix (defined in \texttt{sparsemat.cc}, see Chapter~\ref{ch:sparse}); with \code{treat_as_symmetric} only one
 triangle is assumed to be stored.}
@@ -172,7 +174,7 @@ real and imaginary parts are processed separately.}
 \lafunc{const NRVec operator*(const NRMat<T> &mat) const}{Row vector times matrix, $x\Tr A$ (gemv with \code{'t'}; no conjugation).
 For $A x$ see \code{NRMat::operator*} in Chapter~\ref{ch:mat}.}
 \lafunc{const NRVec operator*(const NRSMat<T> &mat) const}{Row vector times packed symmetric/hermitian matrix, $x\Tr S$ (for hermitian $S$ computed as $S\Tr x = S^* x$ by \code{gemv} with \code{'t'}).}
-\lafunc{const NRVec operator*(const SparseMat<T> &mat) const}{Row vector times sparse matrix; CPU only.}
+\lafunc{const NRVec operator*(const SparseMat<T> &mat) const; const NRVec operator*(const CSRMat<T> &mat) const}{Row vector times sparse matrix; CPU only.}
 \lafunc{const NRMat<T> otimes(const NRVec<T> &rhs, const bool conjugate = false, const T &scale = 1) const}{Outer product
 $A_{ij}=\mathrm{scale}\cdot x_i\, y_j$, with $y_j^*$ if \code{conjugate} (complex only); BLAS ger/geru/gerc.}
 \lafunc{const NRMat<T> operator|(const NRVec<T> &rhs) const}{Same as \code{otimes(rhs,true)}, i.e.\ $x y\Hc$.}

fourindex.cc

@@ -32,6 +32,19 @@ while((unsigned char)i!=terminator&&(unsigned char)j!=terminator&&(unsigned char
         }
 }
 
+template <>
+void fourindex<unsigned char,float>::fscanf(FILE *ff)
+{
+int i,j,k,l;
+float elem;
+if(5!= ::fscanf(ff,"%f %d %d %d %d",&elem,&i,&j,&k,&l)) laerror("read error in fourindex::fscanf");
+while((unsigned char)i!=terminator&&(unsigned char)j!=terminator&&(unsigned char)k!=terminator&&(unsigned char)l!=terminator) //cast: -1 in the file matches the default terminator
+        {
+        add(i,j,k,l,elem);
+        if(5!= ::fscanf(ff,"%f %d %d %d %d",&elem,&i,&j,&k,&l)) laerror("read error in fourindex::fscanf");
+        }
+}
+
 template <>
 void fourindex<unsigned char,double>::fprintf(FILE *f, char *format) const
 {
@@ -44,6 +57,18 @@ while(it!=end)
 ::fprintf(f,format,0.,(int)terminator,(int)terminator,(int)terminator,(int)terminator); //zero line for terminator 0 (FCIDUMP-like)
 }
 
+template <>
+void fourindex<unsigned char,float>::fprintf(FILE *f, char *format) const
+{
+fourindex<unsigned char,float>::iterator it=this->begin(),end=this->end();
+while(it!=end)
+            {
+	    ::fprintf(f,format,it->elem,it->index.indiv.i,it->index.indiv.j,it->index.indiv.k,it->index.indiv.l);
+            ++it;
+            }
+::fprintf(f,format,0.,(int)terminator,(int)terminator,(int)terminator,(int)terminator); //zero line for terminator 0 (FCIDUMP-like)
+}
+
 
 
 /***************************************************************************//**
@@ -52,4 +77,7 @@ while(it!=end)
 template class fourindex<unsigned char,double>;
 template class fourindex_ext<unsigned char,double>;
 template class fourindex_dense<nosymmetry,double,unsigned char>;
+template class fourindex<unsigned char,float>;
+template class fourindex_ext<unsigned char,float>;
+template class fourindex_dense<nosymmetry,float,unsigned char>;
 }

gmres.h

@@ -73,7 +73,7 @@ for(int i=0;i<=k;i++)
 
 //x contains ev. initial guess and on return the solution
 template<typename T, typename Matrix>
-bool gmres(const Matrix &bigmat, const NRVec<T> &b, NRVec<T> &x, const bool doguess=1, const double eps=1e-7, const int MAXIT=50, const bool verbose=1, bool square=1,const bool precondition=1, int neustart=0, const int incore=1)
+bool gmres(const Matrix &bigmat, const NRVec<T> &b, NRVec<T> &x, const bool doguess=1, const double eps=LA_precision<typename LA_traits<T>::normtype>::gmres_eps(), const int MAXIT=50, const bool verbose=1, bool square=1,const bool precondition=1, int neustart=0, const int incore=1)
 {
 int zeilen=bigmat.nrows();
 int spalten=bigmat.ncols();

la_traits.h

@@ -308,6 +308,7 @@ static inline C realpart(const std::complex<C> &x) {return x.real();}
 static inline C imagpart(const std::complex<C> &x) {return x.imag();}
 static inline void setrealpart(std::complex<C> &x, const C &y) {reinterpret_cast<C(&)[2]>(x)[0]=y;}
 static inline void setimagpart(std::complex<C> &x, const C &y) {reinterpret_cast<C(&)[2]>(x)[1]=y;}
+static int strassen_cutoff() {return 2048;} //NRMat::strassen uses plain gemm for blocks with a dimension <= cutoff
 };
 
 
@@ -368,6 +369,7 @@ static inline C realpart(const C &x) {return x;}
 static inline C imagpart(const C &x) {return 0;}
 static inline void setrealpart(C &x, const C &y) {x=y;}
 static inline void setimagpart(C &x, const C &y) {}
+static int strassen_cutoff() {return 2048;} //NRMat::strassen uses plain gemm for blocks with a dimension <= cutoff
 };
 
 
@@ -456,6 +458,34 @@ generate_traits_smat(NRSMat_from1)
 template<typename C>
 struct LA_traits : LA_traits_aux<C, typename isscalar<C>::scalar_type> {};
 
+
+//precision-dependent default thresholds; only float and double are supported (the double values are those used so far)
+template<typename T> struct LA_precision;
+template<> struct LA_precision<double>
+	{
+	static double svd_threshold() {return 1e-12;} //singular values treated as zero (svd_solve, Tucker)
+	static double taylor_precision() {return 1e-14;} //truncation of the Taylor series in matexp
+	static double conjgrad_tol() {return 1e-8;} //default convergence threshold of conjgrad
+	static double gmres_eps() {return 1e-7;} //default relative residual of gmres
+	static double eigensolver_eps() {return 1e-6;} //default convergence threshold of davidson and lanczos
+	static double newton_threshold() {return 1e-14;} //convergence of Polynomial::newton
+	static double det_tiny() {return 1e-300;} //determinant underflow guard in linear_solve
+	static double sparse_epsilon() {return 1e-15;} //elements treated as zero in the sparse matrices (SPARSEEPSILON)
+	static double cholesky_epsilon() {return 1e-16;} //pivot threshold of the sparse Cholesky (CHOLESKYEPSILON)
+	};
+template<> struct LA_precision<float>
+	{
+	static float svd_threshold() {return 1e-5f;}
+	static float taylor_precision() {return 1e-7f;}
+	static float conjgrad_tol() {return 1e-5f;}
+	static float gmres_eps() {return 1e-5f;}
+	static float eigensolver_eps() {return 1e-4f;}
+	static float newton_threshold() {return 1e-6f;}
+	static float det_tiny() {return 1e-37f;}
+	static float sparse_epsilon() {return 1e-7f;}
+	static float cholesky_epsilon() {return 1e-8f;}
+	};
+
 }//namespace
 
 #endif

laerror.cc

@@ -149,7 +149,7 @@ throw LAerror(s1);
 }
 
 
-//stub for f77 blas called from strassen routine
+//stub for f77 blas error handler
 extern "C" void xerbla_(const char name[6], int *n){
 	char msg[1024];
 	strcpy(msg,"LAPACK or BLAS error in routine ");

lanczos.h

@@ -39,7 +39,7 @@ namespace LA {
 
 template <typename T, typename Matrix>
 extern void lanczos(const Matrix &bigmat, NRVec<T> &eivals, NRVec<T> *eivecs, const char *eivecsfile, 
-		int nroots=1,  const bool verbose=0, const double eps=1e-6,
+		int nroots=1,  const bool verbose=0, const double eps=LA_precision<typename LA_traits<T>::normtype>::eigensolver_eps(),
 	 	const bool incore=1, int maxit=100, const int maxkrylov = 1000,
 		void (*initguess)(NRVec<T> &)=NULL, const typename LA_traits<T>::normtype *target=NULL)
 {

mat.h

@@ -424,9 +424,13 @@ public:
 	NRMat & operator+=(const SparseMat<T> &rhs);
 	//! subtract given sparse matrix
         NRMat & operator-=(const SparseMat<T> &rhs);
+	NRMat & operator+=(const CSRMat<T> &rhs);
+	NRMat & operator-=(const CSRMat<T> &rhs);
 
 	//! perform the \b gemm operation
 	void gemm(const T &beta, const SparseMat<T> &a, const char transa, const NRMat &b, const char transb, const T &alpha);
+	void gemm(const T &beta, const CSRMat<T> &a, const char transa, const NRMat &b, const char transb, const T &alpha);
+	const NRMat operator*(const CSRMat<T> &rhs) const; //dense times compressed row sparse, dense result
 
         inline void simplify() {};
 	bool issymmetric() const { return 0; };
@@ -438,11 +442,8 @@ public:
         void printsorted(std::ostream &s, int direction=1, bool absvalue=false, bool stable=false, typename LA_traits<T>::normtype thr=0, int topmax=0) const;
 
 
-#ifndef NO_STRASSEN
+	//! Strassen-Winograd multiplication (\f$\mathcal{O}(n^{2.81})\f$), the same operation as \see NRMat<T>::gemm(); blocks with a dimension <= cutoff are multiplied by gemm
-	//! Strassen's multiplication (better than \f$\mathacal{O}(n^3)\f$, analogous syntax to \see NRMat<T>::gemm() )
+	void strassen(const T &beta, const NRMat &a, const char transa, const NRMat &b, const char transb, const T &alpha, const int cutoff=LA_traits<T>::strassen_cutoff());
-	void strassen(const T beta, const NRMat &a, const char transa, const NRMat &b, const char transb, const T alpha);
-	void s_cutoff(const int,const int,const int,const int) const;
-#endif
 	void swap(NRMat &rhs) //more efficient swap than via tmp and constructors and operator=
 		{
 		int tmpnn=nn; nn=rhs.nn; rhs.nn=tmpnn;
@@ -970,6 +971,27 @@ inline const double NRMat<double>::amax() const{
 #endif
 }
 
+template<>
+inline const float NRMat<float>::amax() const{
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+	#ifdef MATPTR
+		return v[0][cblas_isamax(nn*mm, v[0], 1) ];
+	#else
+		return v[cblas_isamax(nn*mm, v, 1) ];
+	#endif
+#ifdef CUDALA
+	}else{
+		float ret(0.0);
+		const size_t pozice = cublasIsamax((size_t)nn*mm, v, 1);
+		TEST_CUBLAS("cublasIsamax");
+		gpuget(1, sizeof(float), v + pozice, &ret);
+		return ret;
+	}
+#endif
+}
+
 /***************************************************************************//**
  * for this real matrix \f$A\f$, determine the first element
  * with smallest absolute value
@@ -1009,6 +1031,40 @@ inline const double NRMat<double>::amin() const{
 	return ret;
 }
 
+template<>
+inline const float NRMat<float>::amin() const{
+	float ret(0.0);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		// idamin seems not to be supported
+		const size_t nm = (size_t)nn*mm;
+		float val(0.0);
+		int index(-1);
+		ret = std::numeric_limits<float>::max();
+		for(register int i=0; i < nm; i++){
+		#ifdef MATPTR			
+			val = std::abs(v[0][i]);
+		#else
+			val = std::abs(v[i]);
+		#endif
+			if(val < ret){ index = i; ret = val; }
+		}
+		#ifdef MATPTR
+			ret = v[0][index];
+		#else
+			ret = v[index];
+		#endif	
+#ifdef CUDALA
+	}else{
+		const size_t pozice = cublasIsamin((size_t)nn*mm, v, 1);
+		TEST_CUBLAS("cublasIsamin");
+		gpuget(1, sizeof(float), v + pozice, &ret);
+	}
+#endif
+	return ret;
+}
+
 /***************************************************************************//**
  * for this complex matrix \f$A\f$, determine the smallest index of the maximum 
  * magnitude element, i.e. maximal element in the 1-norm
@@ -1036,6 +1092,28 @@ inline const std::complex<double> NRMat<std::complex<double> >::amax() const{
 
 }
 
+template<>
+inline const std::complex<float> NRMat<std::complex<float> >::amax() const{
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+	#ifdef MATPTR
+		return v[0][cblas_icamax(nn*mm, v[0], 1) ];
+	#else
+		return v[cblas_icamax(nn*mm, v, 1) ];
+	#endif
+#ifdef CUDALA
+	}else{
+		std::complex<float> ret(0.0, 0.0);
+		const size_t pozice = cublasIcamax((size_t)nn*mm, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasIcamax");
+		gpuget(1, sizeof(std::complex<float>), v + pozice, &ret);
+		return ret;
+	}
+#endif
+
+}
+
 /***************************************************************************//**
  * for this complex matrix \f$A\f$, determine the smallest index of the minimum 
  * magnitude element, i.e. minimal element in the 1-norm
@@ -1078,6 +1156,43 @@ inline const std::complex<double> NRMat<std::complex<double> >::amin() const{
 	return ret;
 }
 
+template<>
+inline const std::complex<float> NRMat<std::complex<float> >::amin() const{
+	std::complex<float> ret(0.0, 0.0);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		// idamin seems not to be supported
+		const size_t nm = (size_t)nn*mm;
+		int index(-1);
+		float val(0.0), min_val(0.0);
+		std::complex<float> z_val(0.0, 0.0);
+
+		min_val = std::numeric_limits<float>::max();
+		for(register int i=0; i < nm; i++){
+		#ifdef MATPTR			
+			z_val = v[0][i];
+		#else
+			z_val = v[i];
+		#endif
+			val = std::abs(z_val.real()) + std::abs(z_val.imag());
+			if(val < min_val){ index = i; min_val = val; }
+		}
+		#ifdef MATPTR
+			ret = v[0][index];
+		#else
+			ret = v[index];
+		#endif	
+#ifdef CUDALA
+	}else{
+		const size_t pozice = cublasIcamin((size_t)nn*mm, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasIcamin");
+		gpuget(1, sizeof(std::complex<float>), v + pozice, &ret);
+	}
+#endif
+	return ret;
+}
+
 /***************************************************************************//**
  * destructor for general type
  * @see NRMat<T>::count 
@@ -1340,6 +1455,10 @@ NRMat<std::complex<T> > complexify(const NRMat<T> &rhs) {
 	return r;
 }
 
+//BLAS versions for double and float are in mat.cc
+template<> NRMat<std::complex<double> > complexify<double>(const NRMat<double> &rhs);
+template<> NRMat<std::complex<float> > complexify<float>(const NRMat<float> &rhs);
+
 //this is general for any type, complexmatrix() uses blas for doubles
 template<typename T>
 NRMat<std::complex<T> > complexify(const NRMat<T> &rhsr, const NRMat<T> &rhsi) {

matexp.h

@@ -274,7 +274,7 @@ double scale=std::exp(-std::log(2.)*power);
 
 
 //find how long taylor expansion will be necessary
-const double precision=1e-14; //further decreasing brings nothing
+const double precision=LA_precision<typename LA_traits<C>::normtype>::taylor_precision(); //further decreasing brings nothing
 double s,t;
 s=mnorm*scale;
 int n=0;
@@ -311,7 +311,7 @@ if(maxpower>=0 && power>maxpower) power=maxpower;
 double scale=std::exp(-std::log(2.)*power);
 
 //find how long taylor expansion will be necessary
-const double precision=1e-14; //further decreasing brings nothing
+const double precision=LA_precision<typename LA_traits<C>::normtype>::taylor_precision(); //further decreasing brings nothing
 double s,t;
 s=mnorm*scale;
 int n=0;

noncblas.cc

@@ -185,6 +185,18 @@ void cblas_zcopy(const int N, const void *X, const int incX, void *Y, const int
 #endif
 }
 
+extern "C" void FORNAME(ccopy) (const FINT *n, const void *x, const FINT *incx, void *y, const FINT *incy);
+void cblas_ccopy(const int N, const void *X, const int incX, void *Y, const int incY){
+#ifdef FORINT
+	const FINT ntmp=N;
+	const FINT incxtmp=incX;
+	const FINT incytmp=incY;
+	FORNAME(ccopy) (&ntmp,X,&incxtmp,Y,&incytmp);
+#else
+	FORNAME(ccopy) (&N,X,&incX,Y,&incY);
+#endif
+}
+
 extern "C" void FORNAME(zaxpy) (const FINT *n, const void *a, const void *x, const FINT *incx, void *y, const FINT *incy);
 void cblas_zaxpy(const int N, const void *alpha, const void *X, const int incX, void *Y, const int incY){
 #ifdef FORINT
@@ -197,6 +209,18 @@ void cblas_zaxpy(const int N, const void *alpha, const void *X, const int incX,
 #endif
 }
 
+extern "C" void FORNAME(caxpy) (const FINT *n, const void *a, const void *x, const FINT *incx, void *y, const FINT *incy);
+void cblas_caxpy(const int N, const void *alpha, const void *X, const int incX, void *Y, const int incY){
+#ifdef FORINT
+	const FINT ntmp=N;
+	const FINT incxtmp=incX;
+	const FINT incytmp=incY;
+	FORNAME(caxpy) (&ntmp,alpha,X,&incxtmp,Y,&incytmp);
+#else
+	FORNAME(caxpy) (&N,alpha,X,&incX,Y,&incY);
+#endif
+}
+
 extern "C" void FORNAME(zscal) (const FINT *n, const void *a, void *x, const FINT *incx);
 void cblas_zscal(const int N, const void *alpha, void *X, const int incX){
 #ifdef FORINT
@@ -208,6 +232,17 @@ void cblas_zscal(const int N, const void *alpha, void *X, const int incX){
 #endif
 }
 
+extern "C" void FORNAME(cscal) (const FINT *n, const void *a, void *x, const FINT *incx);
+void cblas_cscal(const int N, const void *alpha, void *X, const int incX){
+#ifdef FORINT
+	const FINT ntmp=N;
+	const FINT incxtmp=incX;
+	FORNAME(cscal)(&ntmp,alpha,X,&incxtmp);
+#else
+	FORNAME(cscal)(&N,alpha,X,&incX);
+#endif
+}
+
 extern "C" void FORNAME(zdscal) (const FINT *n, const double *a, void *x, const FINT *incx);
 void cblas_zdscal(const int N, const double alpha, void *X, const int incX){
 #ifdef FORINT
@@ -219,6 +254,17 @@ void cblas_zdscal(const int N, const double alpha, void *X, const int incX){
 #endif
 }
 
+extern "C" void FORNAME(csscal) (const FINT *n, const float *a, void *x, const FINT *incx);
+void cblas_csscal(const int N, const float alpha, void *X, const int incX){
+#ifdef FORINT
+	const FINT ntmp=N;
+	const FINT incxtmp=incX;
+	FORNAME(csscal)(&ntmp,&alpha,X,&incxtmp);
+#else
+	FORNAME(csscal)(&N,&alpha,X,&incX);
+#endif
+}
+
 
 extern "C" double FORNAME(dznrm2) (const FINT *n, const void *x, const FINT *incx);
 double cblas_dznrm2(const int N, const void *X, const int incX){
@@ -294,6 +340,54 @@ void cblas_zdotc_sub(const int N, const void *X, const int incX, const void *Y,
 }
 
 
+#ifdef F2C_COMPLEX_RETURN
+extern "C" void FORNAME(cdotu) (void *retval, const FINT *n, const void *x, const FINT *incx, const void *y, const FINT *incy);
+extern "C" void FORNAME(cdotc) (void *retval, const FINT *n, const void *x, const FINT *incx, const void *y, const FINT *incy);
+#else
+extern "C" std::complex<float> FORNAME(cdotu) (const FINT *n, const void *x, const FINT *incx, const void *y, const FINT *incy);
+extern "C" std::complex<float> FORNAME(cdotc) (const FINT *n, const void *x, const FINT *incx, const void *y, const FINT *incy);
+#endif
+
+void cblas_cdotu_sub(const int N, const void *X, const int incX, const void *Y, const int incY, void *dotu){
+#ifdef FORINT
+	const FINT ntmp=N;
+	const FINT incxtmp=incX;
+	const FINT incytmp=incY;
+#ifdef F2C_COMPLEX_RETURN
+	FORNAME(cdotu) (dotu,&ntmp,X,&incxtmp,Y,&incytmp);
+#else
+	*(std::complex<float> *)dotu = FORNAME(cdotu) (&ntmp,X,&incxtmp,Y,&incytmp);
+#endif
+#else
+#ifdef F2C_COMPLEX_RETURN
+	FORNAME(cdotu) (dotu,&N,X,&incX,Y,&incY);
+#else
+	*(std::complex<float> *)dotu = FORNAME(cdotu) (&N,X,&incX,Y,&incY);
+#endif
+#endif
+}
+
+
+void cblas_cdotc_sub(const int N, const void *X, const int incX, const void *Y, const int incY, void *dotc){
+#ifdef FORINT
+	const FINT ntmp=N;
+	const FINT incxtmp=incX;
+	const FINT incytmp=incY;
+#ifdef F2C_COMPLEX_RETURN
+	FORNAME(cdotc) (dotc,&ntmp,X,&incxtmp,Y,&incytmp);
+#else
+	*(std::complex<float> *)dotc = FORNAME(cdotc) (&ntmp,X,&incxtmp,Y,&incytmp);
+#endif
+#else
+#ifdef F2C_COMPLEX_RETURN
+	FORNAME(cdotc) (dotc,&N,X,&incX,Y,&incY);
+#else
+	*(std::complex<float> *)dotc = FORNAME(cdotc) (&N,X,&incX,Y,&incY);
+#endif
+#endif
+}
+
+
 //Level 2 and Level 3 on symmetric/hermitian packed matrices - straightforward
 //enum CBLAS_UPLO  {CblasUpper=121, CblasLower=122};
 //enum CBLAS_ORDER {CblasRowMajor=101, CblasColMajor=102 };
@@ -382,6 +476,47 @@ p= (std::complex<double> *) X; for(int i=0; i<N; ++i) {p->imag(-p->imag()); p+=
 p= (std::complex<double> *) Y; for(int i=0; i<N; ++i) {p->imag(-p->imag()); p+= incY;}
 }
 
+extern "C" void FORNAME(chpmv) (const char *uplo, const FINT *n, const void *alpha, const void *ap, const void *x, const FINT *incx, const void *beta, void *y, const FINT *incy);
+void cblas_chpmv(const CBLAS_ORDER Order, const CBLAS_UPLO Uplo,
+		const int N, const void *alpha, const void *Ap,
+		const void *X, const int incX,
+		const void *beta, void *Y, const int incY)
+{
+	char U = BLAS_FORTRANCASE('u');
+	if(Order==CblasColMajor && Uplo==CblasUpper) //native fortran convention, no conjugation needed
+		{
+#ifdef FORINT
+		const FINT ntmp=N;
+		const FINT incxtmp=incX;
+		const FINT incytmp=incY;
+		FORNAME(chpmv) (&U,&ntmp, alpha, Ap, X, &incxtmp, beta, Y, &incytmp);
+#else
+		FORNAME(chpmv) (&U,&N, alpha, Ap, X, &incX, beta, Y, &incY);
+#endif
+		return;
+		}
+	if(Order!=CblasRowMajor) laerror("CblasRowMajor or CblasColMajor+CblasUpper order asserted");
+	if(Uplo!=CblasLower) laerror("CblasLower uplo asserted");
+//row-major lower packed hermitian H is column-major upper packed conj(H), therefore
+//conj(y) = conj(beta) conj(y) + conj(alpha) conj(H) conj(x) is computed by chpmv on conjugated vectors and scalars
+std::complex<float> alphac = std::conj(*(const std::complex<float> *)alpha);
+std::complex<float> betac = std::conj(*(const std::complex<float> *)beta);
+std::complex<float> *p;
+p= (std::complex<float> *) X; for(int i=0; i<N; ++i) {p->imag(-p->imag()); p+= incX;}
+p= (std::complex<float> *) Y; for(int i=0; i<N; ++i) {p->imag(-p->imag()); p+= incY;}
+#ifdef FORINT
+	const FINT ntmp=N;
+	const FINT incxtmp=incX;
+	const FINT incytmp=incY;
+	FORNAME(chpmv) (&U,&ntmp, &alphac, Ap, X, &incxtmp, &betac, Y, &incytmp);
+#else
+	FORNAME(chpmv) (&U,&N, &alphac, Ap, X, &incX, &betac, Y, &incY);
+#endif
+//conjugate back
+p= (std::complex<float> *) X; for(int i=0; i<N; ++i) {p->imag(-p->imag()); p+= incX;}
+p= (std::complex<float> *) Y; for(int i=0; i<N; ++i) {p->imag(-p->imag()); p+= incY;}
+}
+
 
 extern "C" void FORNAME(dspr2) (const char *uplo, const FINT *n, const double *alpha, const double *x, const FINT *incx, const double *y, const FINT *incy, double *ap);
 void cblas_dspr2(const CBLAS_ORDER Order, const CBLAS_UPLO Uplo,
@@ -401,6 +536,24 @@ void cblas_dspr2(const CBLAS_ORDER Order, const CBLAS_UPLO Uplo,
 #endif
 }
 
+extern "C" void FORNAME(sspr2) (const char *uplo, const FINT *n, const float *alpha, const float *x, const FINT *incx, const float *y, const FINT *incy, float *ap);
+void cblas_sspr2(const CBLAS_ORDER Order, const CBLAS_UPLO Uplo,
+		const int N, const float alpha, const float *X,
+		const int incX, const float *Y, const int incY, float *Ap)
+{
+	if(Order!=CblasRowMajor) laerror("CblasRowMajor order asserted");
+	if(Uplo!=CblasLower) laerror("CblasLower uplo asserted");
+	char U = BLAS_FORTRANCASE('u');
+#ifdef FORINT
+	const FINT ntmp=N;
+	const FINT incxtmp=incX;
+	const FINT incytmp=incY;
+	FORNAME(sspr2) (&U,&ntmp, &alpha, X, &incxtmp, Y, &incytmp, Ap);
+#else
+	FORNAME(sspr2) (&U,&N, &alpha, X, &incX, Y, &incY, Ap);
+#endif
+}
+
 
 //row-major lower packed hermitian H is column-major upper packed conj(H), therefore
 //conj(H) += conj(a) conj(x) conj(y)^H + a conj(y) conj(x)^H is computed by zhpr2 with alpha=a and conjugated swapped vectors
@@ -430,6 +583,32 @@ p= (std::complex<double> *) X; for(int i=0; i<N; ++i) {p->imag(-p->imag()); p+=
 if(!same) {p= (std::complex<double> *) Y; for(int i=0; i<N; ++i) {p->imag(-p->imag()); p+= incY;}}
 }
 
+extern "C" void FORNAME(chpr2) (const char *uplo, const FINT *n, const void *alpha, const void *x, const FINT *incx, const void *y, const FINT *incy, void *ap);
+void cblas_chpr2(const CBLAS_ORDER Order, const CBLAS_UPLO Uplo, const int N,
+		const void *alpha, const void *X, const int incX,
+		const void *Y, const int incY, void *Ap)
+{
+	if(Order!=CblasRowMajor) laerror("CblasRowMajor order asserted");
+	if(Uplo!=CblasLower) laerror("CblasLower uplo asserted");
+	char U = BLAS_FORTRANCASE('u');
+bool same = (X==Y && incX==incY);
+//conjugate x and y
+std::complex<float> *p;
+p= (std::complex<float> *) X; for(int i=0; i<N; ++i) {p->imag(-p->imag()); p+= incX;}
+if(!same) {p= (std::complex<float> *) Y; for(int i=0; i<N; ++i) {p->imag(-p->imag()); p+= incY;}}
+#ifdef FORINT
+	const FINT ntmp=N;
+	const FINT incxtmp=incX;
+	const FINT incytmp=incY;
+	FORNAME(chpr2) (&U,&ntmp, alpha, Y, &incytmp, X, &incxtmp, Ap);
+#else
+	FORNAME(chpr2) (&U,&N, alpha, Y, &incY, X, &incX, Ap);
+#endif
+//conjugate back
+p= (std::complex<float> *) X; for(int i=0; i<N; ++i) {p->imag(-p->imag()); p+= incX;}
+if(!same) {p= (std::complex<float> *) Y; for(int i=0; i<N; ++i) {p->imag(-p->imag()); p+= incY;}}
+}
+
 
 //Level 2 and Level 3 on general matrices - take into account the transposed storage of matrices in Fortran and C
 
@@ -516,6 +695,47 @@ void cblas_zgeru(const CBLAS_ORDER Order, const int M, const int N,
 #endif
 }
 
+extern "C" void FORNAME(cgeru)  (const FINT *m, const FINT *n, const void *alpha, const void *x, const FINT *incx, const void *y, const FINT *incy, void *a, const FINT *lda);
+void cblas_cgerc(const CBLAS_ORDER Order, const int M, const int N,
+		const void *alpha, const void *X, const int incX,
+		const void *Y, const int incY, void *A, const int lda)
+{
+	if(Order!=CblasRowMajor) laerror("CblasRowMajor order asserted");
+//conjugate y
+std::complex<float> *p;
+p= (std::complex<float> *) Y; for(int i=0; i<N; ++i) {p->imag(-p->imag()); p+= incY;}
+#ifdef FORINT
+	const FINT mtmp=M;
+        const FINT ntmp=N;
+	const FINT incxtmp=incX;
+	const FINT incytmp=incY;
+	const FINT ldatmp=lda;
+	FORNAME(cgeru) (&ntmp, &mtmp, alpha, Y, &incytmp, X, &incxtmp, A, &ldatmp);
+#else
+	FORNAME(cgeru)(&N, &M, alpha, Y, &incY, X, &incX, A, &lda);
+#endif
+//conjugate y back
+p= (std::complex<float> *) Y; for(int i=0; i<N; ++i) {p->imag(-p->imag()); p+= incY;}
+}
+
+
+void cblas_cgeru(const CBLAS_ORDER Order, const int M, const int N,
+		const void *alpha, const void *X, const int incX,
+		const void *Y, const int incY, void *A, const int lda)
+{
+	if(Order!=CblasRowMajor) laerror("CblasRowMajor order asserted");
+#ifdef FORINT
+	const FINT mtmp=M;
+        const FINT ntmp=N;
+	const FINT incxtmp=incX;
+	const FINT incytmp=incY;
+	const FINT ldatmp=lda;
+	FORNAME(cgeru) (&ntmp, &mtmp, alpha, Y, &incytmp, X, &incxtmp, A, &ldatmp);
+#else
+	FORNAME(cgeru)(&N, &M, alpha, Y, &incY, X, &incX, A, &lda);
+#endif
+}
+
 
 extern "C" void FORNAME(dgemm) (const char *transa, const char *transb, const FINT *m, const FINT *n, const FINT *k, const double *alpha, const double *a, const FINT *lda, const double *b, const FINT *ldb, const double *beta, double *c, const FINT *ldc);
 void cblas_dgemm(const CBLAS_ORDER Order, const CBLAS_TRANSPOSE TransA,
@@ -595,6 +815,32 @@ void cblas_zgemm(const CBLAS_ORDER Order, const CBLAS_TRANSPOSE TransA,
 #endif
 }
 
+extern "C" void FORNAME(cgemm) (const char *transa, const char *transb, const FINT *m, const FINT *n, const FINT *k, const void *alpha, const void *a, const FINT *lda, const void *b, const FINT *ldb, const void *beta, void *c, const FINT *ldc);
+void cblas_cgemm(const CBLAS_ORDER Order, const CBLAS_TRANSPOSE TransA,
+		const CBLAS_TRANSPOSE TransB, const int M, const int N,
+		const int K, const void *alpha, const void *A,
+		const int lda, const void *B, const int ldb,
+		const void *beta, void *C, const int ldc)
+{
+	if(Order!=CblasRowMajor) laerror("CblasRowMajor order asserted");
+	//swap a-b, m-n
+	char transb =  BLAS_FORTRANCASE(TransB==CblasConjTrans?'C':(TransB==CblasNoTrans?'N':'T'));
+	char transa =  BLAS_FORTRANCASE(TransA==CblasConjTrans?'C':(TransA==CblasNoTrans?'N':'T'));
+#ifdef FORINT
+	const FINT mtmp=M;
+	const FINT ntmp=N;
+	const FINT ktmp=K;
+	const FINT ldatmp=lda;
+	const FINT ldbtmp=ldb;
+	const FINT ldctmp=ldc;
+	FORNAME(cgemm) (&transb,&transa,
+			&ntmp, &mtmp, &ktmp, alpha, B, &ldbtmp, A, &ldatmp, beta, C, &ldctmp);
+#else
+	FORNAME(cgemm) ( &transb,&transa,
+			&N, &M, &K, alpha, B, &ldb, A, &lda, beta, C, &ldc);
+#endif
+}
+
 
 extern "C" void FORNAME(dgemv)  (const char *TRANS, const FINT *M, const FINT *N, const double *ALPHA, const double *A,  const FINT *LDA,  const double *X,  const FINT *INCX, const double *BETA, double *Y, const FINT *INCY);
 void cblas_dgemv(const CBLAS_ORDER Order,
@@ -691,6 +937,50 @@ if(TransA == CblasConjTrans)
 #endif
 }
 
+extern "C" void FORNAME(cgemv)  (const char *TRANS, const FINT *M, const FINT *N, const void *ALPHA, const void *A,  const FINT *LDA,  const void *X,  const FINT *INCX, const void *BETA, void *Y, const FINT *INCY);
+void cblas_cgemv(const CBLAS_ORDER Order,
+		const CBLAS_TRANSPOSE TransA, const int M, const int N,
+		const void *alpha, const void *A, const int lda,
+		const void *X, const int incX, const void *beta,
+		void *Y, const int incY)
+{
+	if(Order!=CblasRowMajor) laerror("CblasRowMajor order asserted");
+	//swap n-m and toggle transposition
+	char transa =  BLAS_FORTRANCASE(TransA==CblasNoTrans?'T':'N');
+#ifdef FORINT
+	const FINT mtmp=M;
+	const FINT ntmp=N;
+	const FINT ldatmp=lda;
+	const FINT incxtmp=incX;
+	const FINT incytmp=incY;
+#endif
+if(TransA == CblasConjTrans)
+	{
+	//row-major A is column-major A^T, so A^H = conj(A^T) and
+	//conj(y) = conj(beta) conj(y) + conj(alpha) A^T conj(x) is computed by non-transposed cgemv on conjugated vectors and scalars
+	const int lx = M, ly = N; //lengths of x and y for op(A)=A^H
+	std::complex<float> alphac = std::conj(*(const std::complex<float> *)alpha);
+	std::complex<float> betac = std::conj(*(const std::complex<float> *)beta);
+	std::complex<float> *p;
+	p= (std::complex<float> *) X; for(int i=0; i<lx; ++i) {p->imag(-p->imag()); p+= incX;}
+	p= (std::complex<float> *) Y; for(int i=0; i<ly; ++i) {p->imag(-p->imag()); p+= incY;}
+#ifdef FORINT
+	FORNAME(cgemv) (&transa, &ntmp, &mtmp, &alphac, A, &ldatmp, X, &incxtmp, &betac, Y, &incytmp );
+#else
+	FORNAME(cgemv) (&transa, &N, &M, &alphac, A, &lda, X, &incX, &betac, Y, &incY );
+#endif
+	//conjugate back
+	p= (std::complex<float> *) X; for(int i=0; i<lx; ++i) {p->imag(-p->imag()); p+= incX;}
+	p= (std::complex<float> *) Y; for(int i=0; i<ly; ++i) {p->imag(-p->imag()); p+= incY;}
+	return;
+	}
+#ifdef FORINT
+	FORNAME(cgemv) (&transa, &ntmp, &mtmp, alpha, A, &ldatmp, X, &incxtmp, beta, Y, &incytmp );
+#else
+	FORNAME(cgemv) (&transa, &N, &M, alpha, A, &lda, X, &incX, beta, Y, &incY );
+#endif
+}
+
 extern "C" FINT FORNAME(idamax) (const FINT *N, const double *DX, const FINT *INCX);
 CBLAS_INDEX cblas_idamax(const int N, const double *X, const int incX) {
 #ifdef FORINT
@@ -713,6 +1003,17 @@ CBLAS_INDEX cblas_izamax(const int N, const void *X, const int incX) {
 #endif
 }
 
+extern "C" FINT FORNAME(icamax) (const FINT *N, const void *DX, const FINT *INCX);
+CBLAS_INDEX cblas_icamax(const int N, const void *X, const int incX) {
+#ifdef FORINT
+	const FINT ntmp=N;
+	const FINT incxtmp=incX;
+	return (CBLAS_INDEX)FORNAME(icamax)(&ntmp, X, &incxtmp)-1;
+#else
+	return (CBLAS_INDEX)FORNAME(icamax)(&N, X, &incX)-1;
+#endif
+}
+
 extern "C" FINT FORNAME(isamax) (const FINT *N, const float *DX, const FINT *INCX);
 CBLAS_INDEX cblas_isamax(const int N, const float *X, const int incX) {
 #ifdef FORINT
@@ -788,7 +1089,6 @@ int clapack_sgesv(const CBLAS_ORDER Order, const int N, const int NRHS,
 		float *A, const int lda, int *ipiv,
 		float *B, const int ldb)
 {
-std::cout <<"In my clapack_sgesv\n";
 	FINT INFO=0;
 	if(Order!=CblasRowMajor) laerror("CblasRowMajor order asserted");
 	//B should be in the same physical order, just transpose A in place and the LU result on output
@@ -848,4 +1148,16 @@ void cblas_zswap(const int N, void *X, const int incX, void *Y, const int incY){
 #endif
 }
 
+extern "C" void FORNAME(cswap)(const FINT *N, void *X, const FINT *incX, void *Y, const FINT *incY);
+void cblas_cswap(const int N, void *X, const int incX, void *Y, const int incY){
+#ifdef FORINT
+	const FINT N_tmp = N;
+	const FINT incX_tmp = incX;
+	const FINT incY_tmp = incY;
+	FORNAME(cswap)(&N_tmp, X, &incX_tmp, Y, &incY_tmp);
+#else
+	FORNAME(cswap)(&N, X, &incX, Y, &incY);
+#endif
+}
+
 #endif

noncblas.h

@@ -689,5 +689,8 @@ int cblas_errprn(int ierr, int info, char *form, ...);
 int clapack_dgesv(const  CBLAS_ORDER Order, const int N, const int NRHS,
                   double *A, const int lda, int *ipiv,
                   double *B, const int ldb); 
+int clapack_sgesv(const  CBLAS_ORDER Order, const int N, const int NRHS,
+                  float *A, const int lda, int *ipiv,
+                  float *B, const int ldb); 
 #endif
 #endif

nonclass.h

@@ -164,16 +164,18 @@ extern void singular_decomposition(NRMat<T> &a, NRMat<T> *u, NRVec<LA_traits<T>:
 
 declare_la(double)
 declare_la(std::complex<double>)
+declare_la(float)
+declare_la(std::complex<float>)
 
 
 
 //svd_solve without contamination from nullspace, b is n x nrhs, result returns in b, A is destroyed
 template<typename T>
-NRVec<T> svd_solve(NRMat<T> &a, const NRVec<T> &b, double thres=1e-12, int *nullity = NULL)
+NRVec<T> svd_solve(NRMat<T> &a, const NRVec<T> &b, typename LA_traits<T>::normtype thres=LA_precision<typename LA_traits<T>::normtype >::svd_threshold(), int *nullity = NULL)
 {
 if(b.size()!=a.nrows()) laerror("size mismatch in svd_solve");
 if(nullity) *nullity=0;
-NRVec<double> w(a.ncols());
+NRVec<typename LA_traits<T>::normtype > w(a.ncols());
 NRMat<T> u(a.nrows(),a.ncols());
 NRMat<T> vt(a.ncols(),a.ncols());
 singular_decomposition(a,&u,w,&vt,false);
@@ -191,11 +193,11 @@ return res;
 
 
 template<typename T>
-NRMat<T> svd_solve(NRMat<T> &a, const NRMat<T> &b, double thres=1e-12, int *nullity = NULL)
+NRMat<T> svd_solve(NRMat<T> &a, const NRMat<T> &b, typename LA_traits<T>::normtype thres=LA_precision<typename LA_traits<T>::normtype >::svd_threshold(), int *nullity = NULL)
 {
 if(b.nrows()!=a.nrows()) laerror("size mismatch in svd_solve");
 if(nullity) *nullity=0;
-NRVec<double> w(a.ncols());
+NRVec<typename LA_traits<T>::normtype > w(a.ncols());
 NRMat<T> u(a.nrows(),a.ncols());
 NRMat<T> vt(a.ncols(),a.ncols());
 singular_decomposition(a,&u,w,&vt,false);
@@ -221,10 +223,13 @@ return res;
 extern void gdiagonalize(NRMat<double> &a, NRVec<double> &wr, NRVec<double> &wi,
 		NRMat<double> *vl=NULL, NRMat<double> *vr=NULL, const bool corder=1, int n=0, const int sorttype=0, const int biorthonormalize=0,
 		NRMat<double> *b=NULL, NRVec<double> *beta=NULL); //this used real storage of eigenvectors like dgeev: for a complex pair (i,i+1) the eigenvector of w_i is Re + i Im = vector i + i*vector i+1
+extern void gdiagonalize(NRMat<float> &a, NRVec<float> &wr, NRVec<float> &wi,
+		NRMat<float> *vl=NULL, NRMat<float> *vr=NULL, const bool corder=1, int n=0, const int sorttype=0, const int biorthonormalize=0,
+		NRMat<float> *b=NULL, NRVec<float> *beta=NULL);
 
 template<typename T>
-extern void gdiagonalize(NRMat<T> &a, NRVec< std::complex<double> > &w,
+extern void gdiagonalize(NRMat<T> &a, NRVec<typename LA_traits<T>::complextype > &w,
-		 NRMat< std::complex<double> >*vl=NULL, NRMat< std::complex<double> > *vr=NULL,
+		 NRMat<typename LA_traits<T>::complextype > *vl=NULL, NRMat<typename LA_traits<T>::complextype > *vr=NULL,
 		 const bool corder=1, int n=0, const int sorttype=0, const int biorthonormalize=0,
 		NRMat<T> *b=NULL, NRVec<T> *beta=NULL); //eigenvectors are stored in complex matrices for T both double and complex
 
@@ -232,9 +237,13 @@ extern void gdiagonalize(NRMat<T> &a, NRVec< std::complex<double> > &w,
 extern void gdiagonalize(NRMat<std::complex<double> > &a, NRVec<double> &wr, NRVec<double> &wi,
                 NRMat<std::complex<double> > *vl=NULL, NRMat<std::complex<double> > *vr=NULL, const bool corder=1, int n=0, const int sorttype=0, const int biorthonormalize=0,
                 NRMat<std::complex<double> > *b=NULL, NRVec<std::complex<double> > *beta=NULL);
+extern void gdiagonalize(NRMat<std::complex<float> > &a, NRVec<float> &wr, NRVec<float> &wi,
+                NRMat<std::complex<float> > *vl=NULL, NRMat<std::complex<float> > *vr=NULL, const bool corder=1, int n=0, const int sorttype=0, const int biorthonormalize=0,
+                NRMat<std::complex<float> > *b=NULL, NRVec<std::complex<float> > *beta=NULL);
 
 //diagonalization of symmetric real tridiagonal matrix
 extern void diagonalize3(NRVec<double> &d, NRVec<double> &d1, NRMat<double> *v, const bool corder=1, int n0=0);
+extern void diagonalize3(NRVec<float> &d, NRVec<float> &d1, NRMat<float> *v, const bool corder=1, int n0=0);
 
 //complex,real,imaginary parts of various entities
 template<typename T>
@@ -251,6 +260,8 @@ extern const typename LA_traits<T>::complextype complexmatrix (const T&, const T
 //Cholesky decomposition
 extern void cholesky(NRMat<double> &a, bool upper=1);
 extern void cholesky(NRMat<std::complex<double> > &a, bool upper=1);
+extern void cholesky(NRMat<float> &a, bool upper=1);
+extern void cholesky(NRMat<std::complex<float> > &a, bool upper=1);
 
 //inverse by means of linear solve, pass by value to preserve argument intact
 template<typename T>
@@ -268,13 +279,13 @@ const NRMat<T> calcinverse(NRMat<T> a, T *det=NULL)
 
 //inverse by means of SVD , pass by value to preserve argument intact
 template<typename T>
-const NRMat<T> calcsvdinverse(NRMat<T> a, double thr=0, bool do_absadjoint=false)
+const NRMat<T> calcsvdinverse(NRMat<T> a, typename LA_traits<T>::normtype thr=0, bool do_absadjoint=false)
 {
         if(a.nrows()!=a.ncols()) laerror("svdinverse() for non-square matrix");
 	int n=a.nrows();
         a.copyonwrite();
 	NRMat<T> u(n,n),v(n,n);
-	NRVec<double> w(n),s(n);
+	NRVec<typename LA_traits<T>::normtype > w(n),s(n);
 	singular_decomposition(a,&u,w,&v,true);
 	if(do_absadjoint) //absadjoint matrix - missing det(U)*det(V) to be full adjoint - a sign or complex phase
 		{
@@ -348,7 +359,7 @@ return determinant_destroy(a);
 //	the info parameter of gesvx is returned (see man dgesvx)
 //------------------------------------------------------------------------------
 template<class T>
-int linear_solve_x(NRMat<T> &A, T *B, const int rhsCount, const int eqCount, const bool eq, const bool saveA, double *rcond);
+int linear_solve_x(NRMat<T> &A, T *B, const int rhsCount, const int eqCount, const bool eq, const bool saveA, typename LA_traits<T>::normtype *rcond);
 
 
 //------------------------------------------------------------------------------
@@ -364,7 +375,7 @@ int linear_solve_x(NRMat<T> &A, T *B, const int rhsCount, const int eqCount, con
 //	the info parameter of dgesvx is returned (see man dgesvx)
 //------------------------------------------------------------------------------
 template<class T>
-int multiply_by_inverse(NRMat<T> &A, NRMat<T> &B, bool useEq=false, double *rcond=NULL);
+int multiply_by_inverse(NRMat<T> &A, NRMat<T> &B, bool useEq=false, typename LA_traits<T>::normtype *rcond=NULL);
 
 
 //general submatrix, INDEX will typically be NRVec<int> or even int*
@@ -414,6 +425,7 @@ return r;
 
 //auxiliary routine to adjust eigenvectors to guarantee real logarithm
 extern void adjustphases(NRMat<double> &v);
+extern void adjustphases(NRMat<float> &v);
 
 
 //declaration of template interface to cblas routines with full options available
@@ -443,6 +455,24 @@ inline double xdot<double>(int n, const double *x, int incx, const double *y, in
 return cblas_ddot(n,x,incx,y,incy);
 }
 
+template<>
+inline void xcopy<float> (int n, const float *x, int incx, float *y, int incy)
+{
+cblas_scopy(n, x, incx, y, incy);
+}
+
+template<>
+inline void xaxpy<float>(int n, const float &a, const float *x, int incx, float *y, int incy)
+{
+cblas_saxpy(n, a, x, incx, y, incy);
+}
+
+template<>
+inline float xdot<float>(int n, const float *x, int incx, const float *y, int incy)
+{
+return cblas_sdot(n,x,incx,y,incy);
+}
+
 
 
 //debugging aid: reconstruct an explicit matrix from the implicit version
@@ -468,12 +498,14 @@ return r;
 
 extern NRMat<double> realmatrixfunction(NRMat<double> a, double (*f) (double)); //a has to by in fact symmetric
 extern NRMat<std::complex<double> > complexmatrixfunction(NRMat<double> a, double (*fre) (double), double (*fim) (double)); //a has to by in fact symmetric
+extern NRMat<float> realmatrixfunction(NRMat<float> a, float (*f) (float)); //a has to by in fact symmetric
+extern NRMat<std::complex<float> > complexmatrixfunction(NRMat<float> a, float (*fre) (float), float (*fim) (float)); //a has to by in fact symmetric
 
 template<typename T> 
-NRMat<T> matrixfunction(NRSMat<T> a, double (*f) (double)) //of symmetric/hermitian matrix
+NRMat<T> matrixfunction(NRSMat<T> a, typename LA_traits<T>::normtype (*f) (typename LA_traits<T>::normtype)) //of symmetric/hermitian matrix
 {
 	int n = a.nrows();
-	NRVec<double> w(n);
+	NRVec<typename LA_traits<T>::normtype > w(n);
 	NRMat<T> v(n, n);
 	diagonalize(a, w, &v, 0);
 
@@ -488,18 +520,18 @@ NRMat<T> matrixfunction(NRSMat<T> a, double (*f) (double)) //of symmetric/hermit
 
 
 template<typename T>
-extern NRMat<T> matrixfunction(NRMat<T> a, std::complex<double> (*f)(const std::complex<double> &)) //of a general real/complex matrix
+extern NRMat<T> matrixfunction(NRMat<T> a, typename LA_traits<T>::complextype (*f)(const typename LA_traits<T>::complextype &)) //of a general real/complex matrix
 {
 	int n = a.nrows();
-	NRVec<std::complex<double> > w(n);
+	NRVec<typename LA_traits<T>::complextype > w(n);
-	NRMat<std::complex<double> > u(n,n),v(n,n);
+	NRMat<typename LA_traits<T>::complextype > u(n,n),v(n,n);
 
 #ifdef debugmf
-NRMat<std::complex<double> > a0=a;
+NRMat<typename LA_traits<T>::complextype > a0=a;
 #endif
 
         gdiagonalize<T>(a, w, &u, &v, false,n,0,false,NULL,NULL);//a gets destroyed, eigenvectors are rows
-        NRVec< std::complex<double> > z = diagofproduct(u, v, 1, 1);
+        NRVec< typename LA_traits<T>::complextype > z = diagofproduct(u, v, 1, 1);
 
 #ifdef debugmf
 std::cout <<"TEST matrixfunction\n"<<w<<u<<v<<z;
@@ -508,7 +540,7 @@ std::cout <<"TEST matrixfunction2 "<< a0*v.transpose(1) - v.transpose(1)*diagona
 std::cout <<"TEST matrixfunction3 "<< u*v.transpose(1)<<diagonalmatrix(z)<<std::endl;
 #endif
 
-NRVec< std::complex<double> > wz(n);
+NRVec< typename LA_traits<T>::complextype > wz(n);
 for (int i=0; i<a.nrows(); i++) wz[i] = w[i]/z[i];
 
 #ifdef debugmf
@@ -518,7 +550,7 @@ std::cout <<"TEST matrixfunction4 "<< a0<< v.transpose(true)*diagonalmatrix(wz)*
         for (int i=0; i<a.nrows(); i++) w[i] = (*f)(w[i])/z[i];
         u.diagmultl(w);
 
-        NRMat< std::complex<double> > r(n, n);
+        NRMat< typename LA_traits<T>::complextype > r(n, n);
         r.gemm(0.0, v, 'c', u, 'n', 1.0);
 	return (NRMat<T>) r; //convert back to real if applicable by the explicit decomplexifying constructor; it is NOT checked to which accuracy the imaginary part is actually zero
 }
@@ -528,6 +560,8 @@ std::cout <<"TEST matrixfunction4 "<< a0<< v.transpose(true)*diagonalmatrix(wz)*
 
 extern std::complex<double> sqrtinv(const std::complex<double> &);
 extern double sqrtinv(const double);
+extern std::complex<float> sqrtinv(const std::complex<float> &);
+extern float sqrtinv(const float);
 
 //functions on matrices
 inline NRMat<double>  sqrt(const NRSMat<double> &a) { return matrixfunction(a,&std::sqrt); }
@@ -541,6 +575,17 @@ extern NRMat<std::complex<double> > exp0(const NRMat<std::complex<double> > &a);
 extern NRMat<std::complex<double> > copytest(const NRMat<std::complex<double> > &a);
 extern NRMat<double> copytest(const NRMat<double> &a);
 extern NRMat<double> exp0(const NRMat<double> &a);
+inline NRMat<float>  sqrt(const NRSMat<float> &a) { return matrixfunction(a,&std::sqrt); }
+inline NRMat<float>  sqrtinv(const NRSMat<float> &a) { return matrixfunction(a,&sqrtinv); }
+inline NRMat<float>  realsqrt(const NRMat<float> &a) { return realmatrixfunction(a,&std::sqrt); }
+inline NRMat<float>  realsqrtinv(const NRMat<float> &a) { return realmatrixfunction(a,&sqrtinv); }
+inline NRMat<float>  log(const NRSMat<float> &a) { return matrixfunction(a,&std::log); }
+extern NRMat<float> log(const NRMat<float> &a);
+extern NRMat<std::complex<float> > log(const NRMat<std::complex<float> > &a);
+extern NRMat<std::complex<float> > exp0(const NRMat<std::complex<float> > &a);
+extern NRMat<std::complex<float> > copytest(const NRMat<std::complex<float> > &a);
+extern NRMat<float> copytest(const NRMat<float> &a);
+extern NRMat<float> exp0(const NRMat<float> &a);
 
 
 }//namespace

optimize.cc

@@ -1124,7 +1124,7 @@ NRSMat<T> A=J.transposedtimes(); //J^T J
 NRVec<T> g=r*J; //J^T r, the gradient of f
 NRVec<T> diag(n);
 for(int i=0; i<n; ++i) diag[i]=((const NRSMat<T> &)A)(i,i);
-T mu=hp.my_tau*diag.amax(), nu=2;
+T mu=hp.my_scaling ? hp.my_tau : hp.my_tau*diag.amax(), nu=2;
 
 for(; budget; ++st.iter)
 	{
@@ -1163,7 +1163,12 @@ for(; budget; ++st.iter)
 		{
 		mu*=nu; nu*=2;
 		const T eps=std::numeric_limits<T>::epsilon();
-		if(h.norm()<=10*eps*(x.norm()+eps)) //no further improvement at machine precision (cf. Madsen, Nielsen & Tingleff eq. 3.16)
+		//no further improvement at machine precision (cf. Madsen, Nielsen & Tingleff eq. 3.16), tested per component to be scale invariant:
+		//h_i^2 <= (10 eps)^2 (x_i^2+eps^2) for all i (with |h|<=10 eps |x| badly scaled parameters stopped too early in float)
+		NRVec<T> hh(h); hh*=h;
+		NRVec<T> xx(x); xx*=x; xx+=eps*eps;
+		hh/=xx;
+		if(hh.amax()<=100*eps*eps)
 			{
 			if(verbose) std::cout <<name<<": step at machine precision\n";
 			st.converged=true;
@@ -2080,6 +2085,7 @@ template optresult<T> firstorder_minimize(const char *name, firstorder_step<T> &
 template optresult<T> trustregion_minimize(const char *name, trustregion_step<T> &stepper, const OptProblem<T> *o, NRVec<T> &x, const trustregion_hyperparameters<T> &hp, hyperparameters<T> *achieved); \
 
 INSTANTIZE(double)
+INSTANTIZE(float)
 
 
 }//namespace

optimize.h

@@ -1,6 +1,6 @@
 /*
     LA: linear algebra C++ interface library
-    Copyright (C) 2021 Jiri Pittner <jiri.pittner@jh-inst.cas.cz> or <jiri@pittnerovi.com>
+    Copyright (C) 2026 Jiri Pittner <jiri.pittner@jh-inst.cas.cz> or <jiri@pittnerovi.com>
     This code has been prepared with the assistance of an AI (Claude Opus 5.5, Anthropic).
 
     This program is free software: you can redistribute it and/or modify
@@ -34,6 +34,37 @@
 
 namespace LA {
 
+//precision-dependent default steps and thresholds of the optimizers; only double and float are supported (the double values are those used so far)
+template<typename T> struct opt_precision;
+template<> struct opt_precision<double>
+	{
+	static double fd_central_step() {return 1e-6;} //central differences (FiniteDifference, ParameterShift), optimum ~ eps^(1/3)
+	static double fd_forward_step() {return 1e-7;} //forward differences (OptLeastSquares, LM), optimum ~ eps^(1/2)
+	static double g_thr() {return 1e-6;} //gradient convergence
+	static double sy_thr() {return 1e-12;} //curvature condition of the quasi-Newton updates
+	static double sr1_thr() {return 1e-8;} //SR1 skipping rule
+	static double update_thr() {return 1e-8;} //trust-region Hessian update skipping rule
+	static double nm_f_thr() {return 1e-8;} //Nelder-Mead stopping
+	static double nm_x_thr() {return 1e-6;}
+	static double powell_f_thr() {return 1e-10;} //Powell stopping
+	static double powell_x_thr() {return 1e-6;}
+	static double rhoend() {return 1e-6;} //final COBYLA trust radius
+	};
+template<> struct opt_precision<float>
+	{
+	static float fd_central_step() {return 5e-3f;}
+	static float fd_forward_step() {return 3e-4f;}
+	static float g_thr() {return 1e-3f;}
+	static float sy_thr() {return 1e-6f;}
+	static float sr1_thr() {return 1e-4f;}
+	static float update_thr() {return 1e-4f;}
+	static float nm_f_thr() {return 1e-5f;}
+	static float nm_x_thr() {return 1e-4f;}
+	static float powell_f_thr() {return 1e-6f;}
+	static float powell_x_thr() {return 1e-4f;}
+	static float rhoend() {return 1e-4f;}
+	};
+
 //base classes with purely virtual functions for multiple inheritance use
 
 //optimization problem - provides function and optional gradient, in multiply inherited classess will store auxiliary data for the function
@@ -102,7 +133,7 @@ template<typename T>
 class OptLeastSquares : public OptProblem<T>, public OptResiduals<T> {
 public:
 T fd_step; //relative step of the forward-difference Jacobian
-OptLeastSquares(const T h=1e-7) : fd_step(h) {};
+OptLeastSquares(const T h=opt_precision<T>::fd_forward_step()) : fd_step(h) {};
 void jacobian_fd(const NRVec<T> &x, const NRVec<T> &r, NRMat<T> &J) const; //Jacobian by forward differences, r = residuals at x
 virtual T funcgrad(const NRVec<T> &x, NRVec<T> *grad) const;
 virtual ~OptLeastSquares() {};
@@ -164,7 +195,7 @@ class ParameterShift : public OptProblemAdapter<T> {
 public:
 const OptTrigonometric<T> *trig;
 T h;
-ParameterShift(const OptProblem<T> *pp, const T hh=1e-6);
+ParameterShift(const OptProblem<T> *pp, const T hh=opt_precision<T>::fd_central_step());
 virtual T funcgrad(const NRVec<T> &x, NRVec<T> *grad) const;
 };
 
@@ -173,7 +204,7 @@ template<typename T>
 class FiniteDifference : public OptProblemAdapter<T> {
 public:
 T h;
-FiniteDifference(const OptProblem<T> *pp, const T hh=1e-6) : OptProblemAdapter<T>(pp), h(hh) {};
+FiniteDifference(const OptProblem<T> *pp, const T hh=opt_precision<T>::fd_central_step()) : OptProblemAdapter<T>(pp), h(hh) {};
 virtual T funcgrad(const NRVec<T> &x, NRVec<T> *grad) const;
 };
 
@@ -207,7 +238,7 @@ template<typename T>
 class gradient_hyperparameters : public hyperparameters<T> {
 public:
 T my_g_thr; //gradient norm convergence threshold
-gradient_hyperparameters() : my_g_thr(1e-6) {};
+gradient_hyperparameters() : my_g_thr(opt_precision<T>::g_thr()) {};
 virtual void setfrom(const hyperparameters<T> &p) {const gradient_hyperparameters *q=dynamic_cast<const gradient_hyperparameters *>(&p); if(q) *this= *q; else hyperparameters<T>::setfrom(p);}
 virtual bool converged(const optstate<T> &s) const {return (s.gnorm>=0 && s.gnorm<my_g_thr) || hyperparameters<T>::converged(s);}
 virtual void setreached(const optstate<T> &s) {hyperparameters<T>::setreached(s); my_g_thr=s.gnorm;}
@@ -379,7 +410,7 @@ template<typename T>
 class quasinewton_hyperparameters : public linesearch_hyperparameters<T> {
 public:
 T my_sy_thr; //skip the BFGS-type update if s.y <= sy_thr*|s|*|y| (curvature condition violated numerically)
-quasinewton_hyperparameters() : my_sy_thr(1e-12) {};
+quasinewton_hyperparameters() : my_sy_thr(opt_precision<T>::sy_thr()) {};
 virtual void setfrom(const hyperparameters<T> &p) {const quasinewton_hyperparameters *q=dynamic_cast<const quasinewton_hyperparameters *>(&p); if(q) *this= *q; else linesearch_hyperparameters<T>::setfrom(p);}
 };
 
@@ -392,7 +423,7 @@ class BFGS_hyperparameters : public quasinewton_hyperparameters<T> {
 public:
 QN_update my_update; //BFGS, DFP or SR1 (Nocedal & Wright eqs. 6.17, 6.15, 6.25); DFP needs an accurate line search (c2~0.1), otherwise it can stall
 T my_sr1_thr; //SR1: skip the update if |(s-Hy).y| <= sr1_thr*|s-Hy|*|y| (cf. Nocedal & Wright eq. 6.26)
-BFGS_hyperparameters() : my_update(QN_BFGS), my_sr1_thr(1e-8) {};
+BFGS_hyperparameters() : my_update(QN_BFGS), my_sr1_thr(opt_precision<T>::sr1_thr()) {};
 virtual void setfrom(const hyperparameters<T> &p) {const BFGS_hyperparameters *q=dynamic_cast<const BFGS_hyperparameters *>(&p); if(q) *this= *q; else quasinewton_hyperparameters<T>::setfrom(p);}
 };
 
@@ -485,7 +516,7 @@ T my_eta; //a step of a minimization is accepted if (actual reduction)/(predicte
 TR_hessian my_hessian;
 int my_hessian_recompute; //exact Hessian computed every k-th iteration and updated in between; 1 = always exact
 T my_update_thr; //threshold for skipping the Hessian updates (curvature condition of BFGS, denominator of SR1)
-trustregion_hyperparameters() : my_delta0(1), my_deltamax(1e3), my_eta(1e-4), my_hessian(TR_AUTO), my_hessian_recompute(1), my_update_thr(1e-8) {};
+trustregion_hyperparameters() : my_delta0(1), my_deltamax(1e3), my_eta(1e-4), my_hessian(TR_AUTO), my_hessian_recompute(1), my_update_thr(opt_precision<T>::update_thr()) {};
 virtual void setfrom(const hyperparameters<T> &p) {const trustregion_hyperparameters *q=dynamic_cast<const trustregion_hyperparameters *>(&p); if(q) *this= *q; else gradient_hyperparameters<T>::setfrom(p);}
 };
 
@@ -553,7 +584,7 @@ public:
 T my_tau; //initial damping mu = tau*max diag(J^T J)
 bool my_scaling; //Marquardt scaling D = diag(J^T J)
 T my_fd_step; //relative step of the forward-difference Jacobian
-LM_hyperparameters() : my_tau(1e-3), my_scaling(false), my_fd_step(1e-7) {};
+LM_hyperparameters() : my_tau(1e-3), my_scaling(false), my_fd_step(opt_precision<T>::fd_forward_step()) {};
 virtual void setfrom(const hyperparameters<T> &p) {const LM_hyperparameters *q=dynamic_cast<const LM_hyperparameters *>(&p); if(q) *this= *q; else gradient_hyperparameters<T>::setfrom(p);}
 };
 
@@ -576,7 +607,7 @@ T my_alpha; //reflection
 T my_gamma; //expansion
 T my_rho; //contraction
 T my_sigma; //shrink
-NelderMead_hyperparameters() : my_initstep(0.1), my_adaptive(true), my_alpha(1), my_gamma(2), my_rho(0.5), my_sigma(0.5) {this->my_maxstep=5000; this->my_f_thr=1e-8; this->my_x_thr=1e-6;};
+NelderMead_hyperparameters() : my_initstep(0.1), my_adaptive(true), my_alpha(1), my_gamma(2), my_rho(0.5), my_sigma(0.5) {this->my_maxstep=5000; this->my_f_thr=opt_precision<T>::nm_f_thr(); this->my_x_thr=opt_precision<T>::nm_x_thr();};
 virtual void setfrom(const hyperparameters<T> &p) {const NelderMead_hyperparameters *q=dynamic_cast<const NelderMead_hyperparameters *>(&p); if(q) *this= *q; else hyperparameters<T>::setfrom(p);}
 };
 
@@ -596,7 +627,7 @@ public:
 T my_initstep; //initial trial step of the line minimizations
 T my_linmin_tol; //relative tolerance of the line minimizations, the absolute one is 0.01*x_thr
 int my_maxlinemin; //max. no. of function evaluations in one line minimization
-Powell_hyperparameters() : my_initstep(0.1), my_linmin_tol(1e-4), my_maxlinemin(50) {this->my_maxstep=1000; this->my_f_thr=1e-10; this->my_x_thr=1e-6;};
+Powell_hyperparameters() : my_initstep(0.1), my_linmin_tol(1e-4), my_maxlinemin(50) {this->my_maxstep=1000; this->my_f_thr=opt_precision<T>::powell_f_thr(); this->my_x_thr=opt_precision<T>::powell_x_thr();};
 virtual void setfrom(const hyperparameters<T> &p) {const Powell_hyperparameters *q=dynamic_cast<const Powell_hyperparameters *>(&p); if(q) *this= *q; else hyperparameters<T>::setfrom(p);}
 };
 
@@ -620,7 +651,7 @@ T my_rhobeg; //initial trust region radius, about one tenth of the expected chan
 T my_rhoend; //final trust region radius, the accuracy of x
 bool my_flexible_radius; //trust region radius delta >= rho revised by the reduction ratio and rho reduced by the rules of NEWUOA
                          //(Powell, Report DAMTP 2004/NA08, eqs. (7.1), (7.6), (6.16)), so that delta can also increase; false = Powell's original COBYLA with delta = rho
-Cobyla_hyperparameters() : my_rhobeg(0.5), my_rhoend(1e-6), my_flexible_radius(true) {this->my_maxstep=100000;};
+Cobyla_hyperparameters() : my_rhobeg(0.5), my_rhoend(opt_precision<T>::rhoend()), my_flexible_radius(true) {this->my_maxstep=100000;};
 virtual void setfrom(const hyperparameters<T> &p) {const Cobyla_hyperparameters *q=dynamic_cast<const Cobyla_hyperparameters *>(&p); if(q) *this= *q; else hyperparameters<T>::setfrom(p);}
 };
 

polynomial.cc

@@ -264,6 +264,8 @@ template class Polynomial<int>;
 template class Polynomial<unsigned int>;
 template class Polynomial<double>;
 template class Polynomial<std::complex<double> >;
+template class Polynomial<float>;
+template class Polynomial<std::complex<float> >;
 
 #define INSTANTIZE(T) \
 template NRMat<T> Sylvester(const Polynomial<T> &p, const Polynomial<T> &q); \
@@ -277,12 +279,15 @@ INSTANTIZE(int)
 INSTANTIZE(unsigned int)
 INSTANTIZE(double)
 INSTANTIZE(std::complex<double>)
+INSTANTIZE(float)
+INSTANTIZE(std::complex<float>)
 
 #define INSTANTIZE2(T) \
 template Polynomial<T> hermite_polynomial(int); \
 
 INSTANTIZE2(int)
 INSTANTIZE2(double)
+INSTANTIZE2(float)
 
 
 

polynomial.h

@@ -86,7 +86,7 @@ public:
 		while(n>0 && MYABS((*this)[n])<=thr) --n;
 		resize(n,true);
 		};
-	void normalize() {if((*this)[degree()]==(T)0) laerror("zero coefficient at highest degree - simplify first"); *this /= (*this)[degree()];};
+	void normalize() {const T lead=((const Polynomial &)*this)[degree()]; if(lead==(T)0) laerror("zero coefficient at highest degree - simplify first"); *this /= lead;};
 	Polynomial shifted(const int shift) const
 		{
 		if(shift==0) return *this;
@@ -154,9 +154,9 @@ public:
 	Polynomial operator/(const Polynomial &rhs) const {Polynomial q,r; polydiv(rhs,q,r); return q;};
 	Polynomial operator%(const Polynomial &rhs) const {Polynomial q,r; polydiv(rhs,q,r); return r;};
 	NRMat<T> companion() const; //matrix which has this characteristic polynomial
-	NRVec<typename LA_traits<T>::complextype> roots() const; //implemented for complex<double> and double only
+	NRVec<typename LA_traits<T>::complextype> roots() const; //implemented for double, float and their complex types only
 	NRVec<T> realroots(const typename LA_traits<T>::normtype thr) const;
-	T newton(const T x0, const typename LA_traits<T>::normtype thr=1e-14, const int maxit=1000) const; //solve root from the guess
+	T newton(const T x0, const typename LA_traits<T>::normtype thr=LA_precision<typename LA_traits<T>::normtype>::newton_threshold(), const int maxit=1000) const; //solve root from the guess
 	Polynomial pow(const int i) const; //integer power
 	Polynomial powx(const int i) const; //substitute x^i for x in the polynomial
 	void binomial(const int n); //(1+x)^n

smat.cc

@@ -167,7 +167,7 @@ const T* NRSMat<T>::diagonalof(NRVec<T> &r, const bool divide, bool cache) const
 	if(divide){
 		for(register int i=0; i<nn; i++){
 			const T a = v[(size_t)i*(i+1)/2+i];
-			if(a != 0.) r[i] /= a;
+			if(a != (T)0) r[i] /= a;
 		}
 	}else{
 	        for(register int i=0; i<nn; i++) r[i] = v[(size_t)i*(i+1)/2+i];
@@ -215,6 +215,26 @@ const NRSMat<double> NRSMat<double>::operator-() const {
 	return result;
 }
 
+template <>
+const NRSMat<float> NRSMat<float>::operator-() const {
+	NRSMat<float> result(nn, getlocation());
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		memcpy(result.v, v, NN2*sizeof(float));
+		cblas_sscal(NN2, -1., result.v, 1);
+#ifdef CUDALA
+	}else{
+		cublasScopy(NN2, v, 1, result.v, 1);
+		TEST_CUBLAS("cublasScopy");
+
+		cublasSscal(NN2, -1., result.v, 1);
+		TEST_CUBLAS("cublasSscal");
+	}
+#endif
+	return result;
+}
+
 /***************************************************************************//**
  * implements unary minus operator for this hermitian matrix
  * @return modified copy of this matrix
@@ -240,6 +260,27 @@ const NRSMat<std::complex<double> > NRSMat<std::complex<double> >::operator-() c
 	return result;
 }
 
+template <>
+const NRSMat<std::complex<float> > NRSMat<std::complex<float> >::operator-() const {
+	NRSMat<std::complex<float> > result(nn, getlocation());
+#ifdef CUDALA
+        if(location == cpu) {
+#endif
+		memcpy(result.v, v, NN2*sizeof(std::complex<float>));
+		cblas_cscal(NN2, &CMONEF, result.v, 1);
+
+#ifdef CUDALA
+        }else{
+		cublasCcopy(NN2, (cuComplex*)v, 1, (cuComplex*)result.v, 1);
+		TEST_CUBLAS("cublasCcopy");
+
+		cublasCscal(NN2, CUMONE, (cuComplex*)result.v, 1);
+		TEST_CUBLAS("cublasCscal");
+	}
+#endif
+	return result;
+}
+
 /***************************************************************************//**
  * @return the sum of the diagonal elements
  ******************************************************************************/
@@ -266,6 +307,16 @@ void NRSMat<double>::randomize(const double &x) {
 	}
 }
 
+template<>
+void NRSMat<float>::randomize(const float &x) {
+	NOT_GPU(*this);
+	copyonwrite();
+
+	for(size_t i=0; i<NN2; ++i){
+		v[i] = x*RANDDOUBLESIGNED();
+	}
+}
+
 /***************************************************************************//**
  * Fill this hermitian matrix with pseudorandom numbers generated from uniform 
  * distribution. The real and imaginary parts are generated independently.
@@ -282,6 +333,18 @@ void NRSMat<std::complex<double> >::randomize(const double &x) {
 	}
 }
 
+template<>
+void NRSMat<std::complex<float> >::randomize(const float &x) {
+	copyonwrite();
+	for(register size_t i=0; i<NN2; ++i) v[i].real(x*RANDDOUBLESIGNED());
+	for(register size_t i=0; i<NN2; ++i) v[i].imag(x*RANDDOUBLESIGNED());
+	for(register int i=0; i<nn; ++i){
+		for(register int j=0; j<=i; ++j){
+		       	if(i == j) v[i*(size_t)(i+1)/2+j].imag(0.); //hermitean
+		}
+	}
+}
+
 /***************************************************************************//**
  * routine for formatted output via lawritemat
  * @param[in] file pointer to <tt>FILE</tt> structure representing the output file
@@ -390,6 +453,30 @@ const NRMat<double> NRSMat<double>::operator*(const NRMat<double> &rhs) const {
 	return result;
 }
 
+template<>
+const NRMat<float> NRSMat<float>::operator*(const NRMat<float> &rhs) const {
+#ifdef DEBUG
+	if(nn != rhs.nrows()) laerror("incompatible dimensions in NRMat<float> NRSMat<float>::operator*(const NRMat<float> &)");
+#endif
+	SAME_LOC(*this, rhs);
+	NRMat<float> result(nn, rhs.ncols(), getlocation());
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		for(register int k = 0; k<rhs.ncols(); k++){
+			cblas_sspmv(CblasRowMajor, CblasLower, nn, 1.0, v, rhs[0] + k, rhs.ncols(), 0.0, result[0] + k, rhs.ncols());
+		}
+#ifdef CUDALA
+	}else{
+		for(register int k = 0; k<rhs.ncols(); k++){
+			cublasSspmv('U', nn, 1.0, v, rhs[0] + k, rhs.ncols(), 0.0, result[0] + k, rhs.ncols());
+			TEST_CUBLAS("cublasSspmv");
+		}
+	}
+#endif
+	return result;
+}
+
 
 /***************************************************************************//**
  * multiply this real double-precision symmetric matrix \f$S\f$ stored in packed form
@@ -425,6 +512,34 @@ NRSMat<std::complex<double> >::operator*(const NRMat<std::complex<double> > &rhs
 	return result;
 }
 
+template<>
+const NRMat<std::complex<float> >
+NRSMat<std::complex<float> >::operator*(const NRMat<std::complex<float> > &rhs) const {
+#ifdef DEBUG
+	if (nn != rhs.nrows()) laerror("incompatible dimensions in NRSMat<std::complex<float> >::operator*(const NRMat<std::complex<float> > &)");
+#endif
+	SAME_LOC(*this, rhs);
+	NRMat<std::complex<float> > result(nn, rhs.ncols(), getlocation());
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		for(register int k=0; k<rhs.ncols(); k++){
+			cblas_chpmv(CblasRowMajor, CblasLower, nn, &CONEF, v, rhs[0]+k, rhs.ncols(), &CZEROF, result[0]+k, rhs.ncols());
+		}
+#ifdef CUDALA
+	}else{
+		for(register int k = 0; k<rhs.ncols(); k++){
+			cublasChpmv('U', nn, 
+					CUONE, (cuComplex*)v, (cuComplex*)(rhs[0] + k), rhs.ncols(),
+					CUZERO, (cuComplex*)(result[0] + k), rhs.ncols());
+
+			TEST_CUBLAS("cublasSspmv");
+		}
+	}
+#endif
+	return result;
+}
+
 /***************************************************************************//**
  * multiply this real double-precision symmetric matrix \f$S\f$ stored in packed form
  * with real double-precision symmetric matrix \f$T\f$
@@ -478,6 +593,54 @@ const NRMat<double> NRSMat<double>::operator*(const NRSMat<double> &rhs) const {
 	return result;
 }
 
+template<>
+const NRMat<float> NRSMat<float>::operator*(const NRSMat<float> &rhs) const {
+#ifdef DEBUG
+	if (nn != rhs.nn) laerror("incompatible dimensions in NRMat<float> NRSMat<float>::operator*(const NRSMat<float> &)");
+#endif
+	NRMat<float> result(0.0, nn, nn);
+	float *p, *q;
+
+	p = v;
+	for (int i=0; i<nn;i++) {
+		q = rhs.v;
+		for (int k=0; k<=i; k++) {
+			cblas_saxpy(k+1, *p++, q, 1, result[i], 1);
+			q += k+1;
+		}
+	}
+
+	p = v;
+	for (int i=0; i<nn;i++) {
+		q = rhs.v+1;
+		for (int j=1; j<nn; j++) {
+			result[i][j] += cblas_sdot(i+1<j ? i+1 : j, p, 1, q, 1);
+			q += j+1;
+		}
+		p += i+1;
+	}
+
+	p = v; 
+	q = rhs.v;
+	for (int i=0; i<nn; i++) {
+		cblas_sger(CblasRowMajor, i, i+1, 1., p, 1, q, 1, result, nn);
+		p += i+1;
+		q += i+1;
+	}
+	
+	q = rhs.v+3;
+	for (int j=2; j<nn; j++) {
+		p = v+1;
+		for (int i=1; i<j; i++) {
+			cblas_saxpy(i, *++q, p, 1, result[0]+j, nn);
+			p += i+1;
+		}
+		q += 2;
+	}
+
+	return result;
+}
+
 
 /***************************************************************************//**
  * multiply this complex double-precision symmetric matrix \f$G\f$ stored in packed form
@@ -497,6 +660,19 @@ NRSMat<std::complex<double> >::operator*(const NRSMat<std::complex<double> > &rh
 	return result;
 }
 
+template<>
+const NRMat<std::complex<float> > 
+NRSMat<std::complex<float> >::operator*(const NRSMat<std::complex<float> > &rhs) const {
+#ifdef DEBUG
+	if (nn != rhs.nn) laerror("incompatible dimensions in NRSMat<std::complex<float> >::operator*(const NRSMat<std::complex<float> > &)");
+#endif
+	SAME_LOC(*this, rhs);
+	NRMat<std::complex<float> > result(nn, nn, getlocation());
+	NRMat<std::complex<float> > rhsmat(rhs);
+	result = *this * rhsmat;
+	return result;
+}
+
 
 /***************************************************************************//**
  * compute inner product of this real symmetric matrix \f$A\f$ with given real symmetric matrix \f$B\f$
@@ -524,6 +700,25 @@ const double NRSMat<double>::dot(const NRSMat<double> &rhs) const {
 	return ret;
 }
 
+template<>
+const float NRSMat<float>::dot(const NRSMat<float> &rhs) const {
+	float ret(0.);
+#ifdef DEBUG
+	if (nn != rhs.nn) laerror("incompatible dimensions in float NRSMat<float>::dot(const NRSMat<float> &)");
+#endif
+	SAME_LOC(*this, rhs);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		ret = cblas_sdot(NN2, v, 1, rhs.v, 1);
+#ifdef CUDALA
+	}else{
+		ret = cublasSdot(NN2, v, 1, rhs.v, 1);
+	}
+#endif
+	return ret;
+}
+
 
 /***************************************************************************//**
  * compute inner product of this complex symmetric matrix \f$A\f$ with given complex symmetric matrix \f$B\f$
@@ -554,6 +749,28 @@ const std::complex<double> NRSMat<std::complex<double> >::dot(const NRSMat<std::
 	return dot;
 }
 
+template<>
+const std::complex<float> NRSMat<std::complex<float> >::dot(const NRSMat<std::complex<float> > &rhs) const {
+#ifdef DEBUG
+	if (nn != rhs.nn) laerror("incompatible dimensions in std::complex<float> NRSMat<std::complex<float> >::dot(const NRSMat<std::complex<float> > &)");
+#endif
+	std::complex<float> dot(0., 0.);
+	SAME_LOC(*this, rhs);
+
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_cdotc_sub(NN2, v, 1, rhs.v, 1, &dot);
+#ifdef CUDALA
+	}else{
+		const cuComplex _dot = cublasCdotc(NN2, (cuComplex*)v, 1, (cuComplex*)(rhs.v), 1);
+		dot = std::complex<float>(cuCreal(_dot), cuCimag(_dot));
+		TEST_CUBLAS("cublasCdotc");
+	}
+#endif
+	return dot;
+}
+
 
 /***************************************************************************//**
  * compute inner product of this real double-precision symmetric matrix \f$S\f$ of order \f$n\f$
@@ -581,6 +798,26 @@ const double NRSMat<double>::dot(const NRVec<double> &rhs) const {
 return ret;
 }
 
+template<>
+const float NRSMat<float>::dot(const NRVec<float> &rhs) const {
+	float ret(0.0);
+#ifdef DEBUG
+	if(NN2 != rhs.nn) laerror("incompatible dimensions in float NRSMat<float>::dot(const NRVec<float> &)");
+#endif
+	SAME_LOC(*this, rhs);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		ret = cblas_sdot(NN2, v, 1, rhs.v, 1);
+#ifdef CUDALA
+	}else{
+		ret = cublasSdot(NN2, v, 1, rhs.v, 1);
+		TEST_CUBLAS("cublasSdot");
+	}
+#endif
+return ret;
+}
+
 
 /***************************************************************************//**
  * compute inner product of this complex double-precision hermitian matrix \f$H\f$ of order \f$n\f$
@@ -610,6 +847,28 @@ NRSMat<std::complex<double> >::dot(const NRVec<std::complex<double> > &rhs) cons
 	return dot;
 }
 
+template<>
+const std::complex<float> 
+NRSMat<std::complex<float> >::dot(const NRVec<std::complex<float> > &rhs) const {
+#ifdef DEBUG
+	if(NN2 != rhs.nn) laerror("incompatible dimensions in std::complex<float>  NRSMat<std::complex<float> >::dot(const NRVec<std::complex<float> > &)");
+#endif
+	std::complex<float> dot(0., 0.);
+	SAME_LOC(*this, rhs);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_cdotc_sub(NN2, v, 1, rhs.v, 1, &dot);
+#ifdef CUDALA
+	}else{
+		const cuComplex _dot = cublasCdotc(NN2, (cuComplex*)v, 1, (cuComplex*)rhs.v, 1);
+		TEST_CUBLAS("cublasCdotc");
+		dot = std::complex<float>(cuCreal(_dot), cuCimag(_dot));
+	}
+#endif
+	return dot;
+}
+
 /***************************************************************************//**
  * compute the Frobenius norm of this real double-precision symmetric  matrix
  * @param[in] scalar subtract this scalar value from the diagonal elements before the norm computation
@@ -645,6 +904,37 @@ const double NRSMat<double>::norm(const double scalar) const {
 	return std::sqrt(sum);
 }
 
+template<>
+const float NRSMat<float>::norm(const float scalar) const {
+	if(!scalar){
+		float ret(0.);
+#ifdef CUDALA
+		if(location == cpu){
+#endif
+			ret = cblas_snrm2(NN2, v, 1);
+#ifdef CUDALA
+		}else{
+			ret = cublasSnrm2(NN2, v, 1);
+			TEST_CUBLAS("cublasSnrm2");
+		}
+#endif
+		return ret;
+	}
+
+	NOT_GPU(*this);
+
+	float sum(0.);
+	int k(0);
+	for(register int i=0; i<nn; ++i){
+		for(register int j=0; j<=i; ++j) {
+			register float tmp = v[k++];
+			if(i == j) tmp -= scalar;
+			sum += tmp*tmp;
+		}
+	}
+	return std::sqrt(sum);
+}
+
 /***************************************************************************//**
  * compute the Frobenius norm of this complex double-precision hermitian  matrix
  * @param[in] scalar subtract this scalar value from the diagonal elements before the norm computation
@@ -680,6 +970,37 @@ const double NRSMat< std::complex<double> >::norm(const std::complex<double> sca
 	return std::sqrt(sum);
 }
 
+template<>
+const float NRSMat< std::complex<float> >::norm(const std::complex<float> scalar) const {
+	if(!(scalar.real()) && !(scalar.imag())){
+		float ret(0.);
+#ifdef CUDALA
+		if(location == cpu){
+#endif
+			ret = cblas_scnrm2(NN2, v, 1);
+#ifdef CUDALA
+		}else{
+			ret = cublasScnrm2(NN2, (cuComplex*)v, 1);
+			TEST_CUBLAS("cublasScnrm2");
+		}
+#endif
+		return ret;
+	}
+
+	int k(0);
+	float sum(0.);
+	std::complex<float> tmp;
+
+	for(register int i=0; i<nn; ++i){
+		for(register int j=0; j<=i; ++j){
+			tmp = v[k++];
+			if (i == j) tmp -= scalar;
+			sum += tmp.real()*tmp.real() + tmp.imag()*tmp.imag();
+		}
+	}
+	return std::sqrt(sum);
+}
+
 
 /***************************************************************************//**
  * symmetric (hermitian) rank-2 update of this matrix stored in packed form
@@ -709,6 +1030,16 @@ void NRSMat<double>::rank2update(const double alpha, const NRVec<double> &x, con
 	cblas_dspr2(CblasRowMajor, CblasLower, nn, alpha, x, 1, y, 1, v);
 }
 
+template<>
+void NRSMat<float>::rank2update(const float alpha, const NRVec<float> &x, const NRVec<float> &y) {
+#ifdef DEBUG
+	if(nn != x.size() || nn != y.size()) laerror("incompatible dimensions in NRSMat<float>::rank2update");
+#endif
+	NOT_GPU(*this); NOT_GPU(x); NOT_GPU(y);
+	copyonwrite();
+	cblas_sspr2(CblasRowMajor, CblasLower, nn, alpha, x, 1, y, 1, v);
+}
+
 template<>
 void NRSMat<std::complex<double> >::rank2update(const std::complex<double> alpha, const NRVec<std::complex<double> > &x, const NRVec<std::complex<double> > &y) {
 #ifdef DEBUG
@@ -719,6 +1050,16 @@ void NRSMat<std::complex<double> >::rank2update(const std::complex<double> alpha
 	cblas_zhpr2(CblasRowMajor, CblasLower, nn, &alpha, x, 1, y, 1, v);
 }
 
+template<>
+void NRSMat<std::complex<float> >::rank2update(const std::complex<float> alpha, const NRVec<std::complex<float> > &x, const NRVec<std::complex<float> > &y) {
+#ifdef DEBUG
+	if(nn != x.size() || nn != y.size()) laerror("incompatible dimensions in NRSMat<std::complex<float> >::rank2update");
+#endif
+	NOT_GPU(*this); NOT_GPU(x); NOT_GPU(y);
+	copyonwrite();
+	cblas_chpr2(CblasRowMajor, CblasLower, nn, &alpha, x, 1, y, 1, v);
+}
+
 
 /***************************************************************************//**
  * for this real double-precision symmetric matrix \f$S\f$ stored in packed form,
@@ -744,6 +1085,25 @@ void NRSMat<double>::axpy(const double alpha, const NRSMat<double> &x) {
 #endif
 }
 
+template<>
+void NRSMat<float>::axpy(const float alpha, const NRSMat<float> &x) {
+#ifdef DEBUG
+	if(nn != x.nn) laerror("incompatible dimensions in void NRSMat<float>::axpy(const float, const NRSMat<float>&)");
+#endif
+	SAME_LOC(*this, x);
+	copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_saxpy(NN2, alpha, x.v, 1, v, 1);
+#ifdef CUDALA
+	}else{
+		cublasSaxpy(NN2, alpha, x.v, 1, v, 1);
+		TEST_CUBLAS("cublasSaxpy");
+	}
+#endif
+}
+
 
 /***************************************************************************//**
  * for this complex double-precision hermitian matrix \f$H\f$ stored in packed form,
@@ -771,6 +1131,27 @@ void NRSMat<std::complex<double> >::axpy(const std::complex<double> alpha, const
 	
 }
 
+template<>
+void NRSMat<std::complex<float> >::axpy(const std::complex<float> alpha, const NRSMat<std::complex<float> > & x) {
+#ifdef DEBUG
+	if(nn != x.nn) laerror("incompatible dimensions in void NRSMat<std::complex<float> >::axpy(const std::complex<float> , const NRSMat<std::complex<float> >&)");
+#endif
+	SAME_LOC(*this, x);
+	copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_caxpy(NN2, &alpha, x.v, 1, v, 1);
+#ifdef CUDALA
+	}else{
+		const cuComplex _alpha = make_cuComplex(alpha.real(), alpha.imag());
+		cublasCaxpy(NN2, _alpha, (cuComplex*)x.v, 1, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasCaxpy");
+	}
+#endif
+	
+}
+
 /***************************************************************************//**
  * create hermitian matrix \f$H\f$ from given real double-precision symmetric
  * matrix \f$S\f$
@@ -803,6 +1184,32 @@ NRSMat<std::complex<double> >::NRSMat(const NRSMat<double> &rhs, bool imagpart):
 #endif
 }
 
+template<>
+NRSMat<std::complex<float> >::NRSMat(const NRSMat<float> &rhs, bool imagpart): nn(rhs.nrows()), count(new int(1)) {
+	//inconsistent in general case?
+	const int nnp1 = nn*(nn + 1)/2;
+#ifdef CUDALA
+	location = rhs.getlocation();
+	if(location == cpu){
+#endif
+		v = new std::complex<float>[nnp1];
+		memset(v, 0, nnp1*sizeof(std::complex<float>));
+		cblas_scopy(nnp1, &rhs(0, 0), 1, ((float *)v) + (imagpart?1:0), 2);
+#ifdef CUDALA
+	}else{
+		v = (std::complex<float>*) gpualloc(nnp1*sizeof(std::complex<float>));
+
+		std::complex<float> *_val = gpuputcomplex(CZEROF);
+		cublasCcopy(nnp1, (cuComplex*)_val, 0, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasCcopy");
+		gpufree(_val);
+
+		cublasScopy(nnp1, (float*)(&rhs(0,0)), 1, ((float*)v) + (imagpart?1:0), 2);
+		TEST_CUBLAS("cublasScopy"); 
+	}
+#endif
+}
+
 
 
 /***************************************************************************//**
@@ -917,9 +1324,15 @@ void NRSMat<T>::storesubmatrix(const NRVec<int> &selection, const NRSMat &rhs)
 template<>
 NRSMat<double>  NRSMat<double>::inverse() {return NRSMat<double>(NRMat<double>(*this).inverse());}
 
+template<>
+NRSMat<float>  NRSMat<float>::inverse() {return NRSMat<float>(NRMat<float>(*this).inverse());}
+
 template<>
 NRSMat<std::complex<double> >  NRSMat<std::complex<double> >::inverse() {return NRSMat<std::complex<double> >(NRMat<std::complex<double> >(*this).inverse());}
 
+template<>
+NRSMat<std::complex<float> >  NRSMat<std::complex<float> >::inverse() {return NRSMat<std::complex<float> >(NRMat<std::complex<float> >(*this).inverse());}
+
 
 
 /***************************************************************************//**
@@ -978,13 +1391,51 @@ NRSMat<std::complex<float> >& NRSMat<std::complex<float> >::conjugateme() {
 
 
 
+/***************************************************************************//**
+ * complexify a given real matrix \f$A\f$ (double and float, by BLAS; the complex default constructor zeroes the imaginary parts)
+ * @param[in] rhs matrix \f$A\f$ intended for this operation
+ * @return matrix \f$B\f$ where \f$\Re B=A\f$ and \f$\Im B = 0\f$
+ ******************************************************************************/
+template<>
+NRSMat<std::complex<double> > complexify<double>(const NRSMat<double> &rhs) {
+	NRSMat<std::complex<double> > r(rhs.nrows(), rhs.getlocation());
+#ifdef CUDALA
+	if(rhs.getlocation() == cpu){
+#endif
+		cblas_dcopy(rhs.size(), &(rhs[0]), 1, (double*)(&(r[0])), 2);
+#ifdef CUDALA
+	}else{
+		cublasDcopy(rhs.size(), &(rhs[0]), 1, (double*)(&(r[0])), 2);
+		TEST_CUBLAS("cublasDcopy");//"NRSMat<std::complex<double> > complexify(const NRSMat<double> &)"
+	}
+#endif
+	return r;
+}
+
+template<>
+NRSMat<std::complex<float> > complexify<float>(const NRSMat<float> &rhs) {
+	NRSMat<std::complex<float> > r(rhs.nrows(), rhs.getlocation());
+#ifdef CUDALA
+	if(rhs.getlocation() == cpu){
+#endif
+		cblas_scopy(rhs.size(), &(rhs[0]), 1, (float*)(&(r[0])), 2);
+#ifdef CUDALA
+	}else{
+		cublasScopy(rhs.size(), &(rhs[0]), 1, (float*)(&(r[0])), 2);
+		TEST_CUBLAS("cublasScopy");//"NRSMat<std::complex<float> > complexify(const NRSMat<float> &)"
+	}
+#endif
+	return r;
+}
+
+
 /***************************************************************************//**
  * forced instantization in the corresponding object file
  ******************************************************************************/
 template class NRSMat<double>;
 template class NRSMat<std::complex<double> >;
-//template class NRSMat<float>;
+template class NRSMat<float>;
-//template class NRSMat<std::complex<float> >;
+template class NRSMat<std::complex<float> >;
 template class NRSMat<long long>;
 template class NRSMat<long>;
 template class NRSMat<int>;

smat.h

@@ -235,6 +235,7 @@ public:
 //members concerning sparse matrix
 	explicit NRSMat(const SparseMat<T> &rhs);               // dense from sparse
 	explicit NRSMat(const SparseSMat<T> &rhs);               // dense from sparse
+	explicit NRSMat(const CSRMat<T> &rhs);               // dense from sparse (lower triangle)
 	inline void simplify() {}; //just for compatibility with sparse ones
 	bool issymmetric() const {return 1;}
 	void swap(NRSMat &rhs) //more efficient swap than via tmp and constructors and operator=
@@ -387,6 +388,22 @@ inline NRSMat<double> & NRSMat<double>::operator*=(const double &a) {
 	return *this;
 }
 
+template<>
+inline NRSMat<float> & NRSMat<float>::operator*=(const float &a) {
+	copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_sscal(NN2, a, v, 1);
+#ifdef CUDALA
+	}else{
+		cublasSscal(NN2, a, v, 1);
+		TEST_CUBLAS("cublasSscal");//"NRSMat<float>& NRSMat<float>::operator*=(const float &)"
+	}
+#endif
+	return *this;
+}
+
 /***************************************************************************//**
  * multiply this complex symmetric matrix with complex scalar value
  * @param[in] a complex multiplicative factor
@@ -410,6 +427,24 @@ NRSMat<std::complex<double> >::operator*=(const std::complex<double> &a) {
 	return *this;
 }
 
+template<>
+inline NRSMat<std::complex<float> > &
+NRSMat<std::complex<float> >::operator*=(const std::complex<float> &a) {
+	copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_cscal(NN2, &a, v, 1);
+#ifdef CUDALA
+	}else{
+		const cuComplex _a = make_cuComplex(a.real(), a.imag());
+		cublasCscal(NN2, _a, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasCscal");//"NRSMat<std::complex<float> >& NRSMat<std::complex<float> >::operator*=(const std::complex<float> &)"
+	}
+#endif
+	return *this;
+}
+
 
 /***************************************************************************//**
  * multiply this symmetric matrix of general type <code>T</code> stored in packed form
@@ -508,6 +543,27 @@ inline NRSMat<double>& NRSMat<double>::operator+=(const NRSMat<double> & rhs) {
 	return *this;
 }
 
+template<>
+inline NRSMat<float>& NRSMat<float>::operator+=(const NRSMat<float> & rhs) {
+#ifdef DEBUG
+	if(nn != rhs.nn) laerror("incompatible dimensions in NRSMat<float>& NRSMat<float>::operator+=(const NRSMat<float> &)");
+#endif
+	SAME_LOC(*this, rhs);
+	copyonwrite();
+
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_saxpy(NN2, 1.0, rhs.v, 1, v, 1);
+#ifdef CUDALA
+	}else{
+		cublasSaxpy(NN2, 1.0, rhs.v, 1, v, 1);
+		TEST_CUBLAS("cublasSaxpy");//" NRSMat<float>& NRSMat<float>::operator+=(const NRSMat<float> &)"
+	}
+#endif
+	return *this;
+}
+
 /***************************************************************************//**
  * add up this complex symmetric matrix with given symmetric matrix
  * @param[in] rhs complex symmetric matrix to be added
@@ -534,6 +590,27 @@ inline NRSMat<std::complex<double> >& NRSMat<std::complex<double> >::operator+=(
 	return *this;
 }
 
+template<>
+inline NRSMat<std::complex<float> >& NRSMat<std::complex<float> >::operator+=(const NRSMat<std::complex<float> > & rhs) {
+#ifdef DEBUG
+	if(nn != rhs.nn) laerror("incompatible dimensions in NRSMat<std::complex<float> >& NRSMat<std::complex<float> >::operator+=(const NRSMat<std::complex<float> > &)");
+#endif
+	SAME_LOC(*this, rhs);
+	copyonwrite();
+
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_caxpy(NN2, &CONEF, rhs.v, 1, v, 1);
+#ifdef CUDALA
+	}else{
+		cublasCaxpy(NN2, CUONE, (cuComplex*)(rhs.v), 1, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasCaxpy");//"NRSMat<std::complex<float> >& NRSMat<std::complex<float> >::operator+=(const NRSMat<std::complex<float> > &)"
+	}
+#endif
+	return *this;
+}
+
 /***************************************************************************//**
  * add up this symmetric matrix of general type <code>T</code> with given symmetric matrix
  * @param[in] rhs complex matrix of general type <code>T</code> to be added
@@ -578,6 +655,27 @@ inline NRSMat<double>& NRSMat<double>::operator-=(const NRSMat<double>& rhs) {
 	return *this;
 }
 
+template<>
+inline NRSMat<float>& NRSMat<float>::operator-=(const NRSMat<float>& rhs) {
+#ifdef DEBUG
+	if(nn != rhs.nn) laerror("incompatible dimensions in NRSMat<float>& NRSMat<float>::operator-=(const NRSMat<float> &)");
+#endif
+	SAME_LOC(*this, rhs);
+	copyonwrite();
+
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_saxpy(NN2, -1.0, rhs.v, 1, v, 1);
+#ifdef CUDALA
+	}else{
+		cublasSaxpy(NN2, -1.0, rhs.v, 1, v, 1);
+		TEST_CUBLAS("cublasSaxpy");//" NRSMat<float>& NRSMat<float>::operator-=(const NRSMat<float> &)"
+	}
+#endif
+	return *this;
+}
+
 /***************************************************************************//**
  * subtracts given complex symmetric matrix from this complex symmetric matrix
  * @param[in] rhs complex symmetric matrix to be subtracted
@@ -604,6 +702,27 @@ inline NRSMat<std::complex<double> >& NRSMat<std::complex<double> >::operator-=(
 	return *this;
 }
 
+template<>
+inline NRSMat<std::complex<float> >& NRSMat<std::complex<float> >::operator-=(const NRSMat<std::complex<float> >& rhs) {
+#ifdef DEBUG
+	if(nn != rhs.nn) laerror("incompatible dimensions in NRSMat<std::complex<float> >& NRSMat<std::complex<float> >::operator-=(const NRSMat<std::complex<float> > &)");
+#endif
+	SAME_LOC(*this, rhs);
+	copyonwrite();
+
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_caxpy(NN2, &CMONEF, rhs.v, 1, v, 1);
+#ifdef CUDALA
+	}else{
+		cublasCaxpy(NN2, CUMONE, (cuComplex*)(rhs.v), 1, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasCaxpy");//"NRSMat<std::complex<float> >& NRSMat<std::complex<float> >::operator-=(const NRSMat<std::complex<float> > &)"
+	}
+#endif
+	return *this;
+}
+
 /***************************************************************************//**
  * subtracts given symmetric matrix of general type <code>T</code> from this symmetric matrix of type <code>T</code>
  * @param[in] rhs symmetric matrix of general type <code>T</code> to be subtracted
@@ -871,6 +990,24 @@ inline const double NRSMat<double>::amax() const {
 	return ret;
 }
 
+template<>
+inline const float NRSMat<float>::amax() const {
+	float ret(0.0);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		ret = v[cblas_isamax(NN2, v, 1)];
+#ifdef CUDALA
+	}else{
+		const int pozice = cublasIsamax(NN2, v, 1) ;
+		TEST_CUBLAS("cublasIsamax");//"float NRSMat<float>::amax()"
+
+		gpuget(1, sizeof(float), v + pozice, &ret);
+	}
+#endif
+	return ret;
+}
+
 /***************************************************************************//**
  * for this real symmetric matrix \f$A\f$, determine the 
  * first element with smallest absolute value
@@ -901,6 +1038,31 @@ inline const double NRSMat<double>::amin() const {
 	return ret;
 }
 
+template<>
+inline const float NRSMat<float>::amin() const {
+	float ret(0.0);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		// idamin seems not to be supported
+		float val(0.0);
+		int index(-1);
+		ret = std::numeric_limits<float>::max();
+		for(register size_t i = 0; i < NN2; i++){
+			val = std::abs(v[i]);
+			if(val < ret){ index = i; ret = val; }
+		}
+		ret = v[index];
+#ifdef CUDALA
+	}else{
+		const int pozice = cublasIsamin(nn, v, 1);
+		TEST_CUBLAS("cublasIsamin");//"float NRSMat<float>::amin()"
+		gpuget(1, sizeof(float), v + pozice, &ret);
+	}
+#endif
+	return ret;
+}
+
 /***************************************************************************//**
  * for this complex symmetric matrix \f$A\f$, determine the 
  * first element with largest "absolute value"
@@ -923,6 +1085,23 @@ inline const std::complex<double> NRSMat< std::complex<double> >::amax() const{
 	return ret;
 }
 
+template<>
+inline const std::complex<float> NRSMat< std::complex<float> >::amax() const{
+	std::complex<float> ret(0., 0.);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		ret = v[cblas_icamax(NN2, v, 1)];
+#ifdef CUDALA
+	}else{
+		const int pozice = cublasIcamax(NN2, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasIcamax");//"std::complex<float> NRSMat<std::complex<float> >::amax()"
+		gpuget(1, sizeof(std::complex<float>), v + pozice, &ret);
+	}
+#endif
+	return ret;
+}
+
 /***************************************************************************//**
  * for this complex symmetric matrix \f$A\f$, determine the 
  * first element with smallest "absolute value"
@@ -956,6 +1135,34 @@ inline const std::complex<double> NRSMat<std::complex<double> >::amin() const{
 	return ret;
 }
 
+template<>
+inline const std::complex<float> NRSMat<std::complex<float> >::amin() const{
+	std::complex<float> ret(0., 0.);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		// izamin seems not to be supported
+		int index(0);
+		float val(0.0), min_val(0.0);
+		std::complex<float> z_val(0.0, 0.0);
+
+		min_val = std::numeric_limits<float>::max();
+		for(register size_t i = 0; i < NN2; i++){
+			z_val = v[i];
+			val = std::abs(z_val.real()) + std::abs(z_val.imag());
+			if(val < min_val){ index = i; min_val = val; }	
+		}
+		ret = v[index];
+#ifdef CUDALA
+	}else{
+		const int pozice = cublasIcamin(nn, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasIcamin");//"std::complex<float> NRSMat<std::complex<float> >::amin()"
+		gpuget(1, sizeof(std::complex<float>), v + pozice, &ret);
+	}
+#endif
+	return ret;
+}
+
 /***************************************************************************//**
  * @return pointer of general type T to the underlying data structure
  ******************************************************************************/
@@ -1215,28 +1422,9 @@ NRSMat<std::complex<T> > complexify(const NRSMat<T> &rhs) {
 }
 
 
-/***************************************************************************//**
+//BLAS versions for double and float are in smat.cc (a full specialization defined in the header would be multiply defined)
- * complexify a given double-precision real matrix \f$A\f$
+template<> NRSMat<std::complex<double> > complexify<double>(const NRSMat<double> &rhs);
- * @param[in] rhs matrix \f$A\f$ intended for this operation
+template<> NRSMat<std::complex<float> > complexify<float>(const NRSMat<float> &rhs);
- * @return matrix \f$B\f$ where \f$\Re B=A\f$ and \f$\Im B = 0\f$
- ******************************************************************************/
-/*
-template<>
-NRSMat<std::complex<double> > complexify(const NRSMat<double> &rhs) {
-	NRSMat<std::complex<double> > r(rhs.nrows(), rhs.getlocation());
-#ifdef CUDALA
-	if(rhs.getlocation() == cpu){
-#endif
-		cblas_dcopy(rhs.size(), &(rhs[0]), 1, (double*)(&(r[0])), 2);
-#ifdef CUDALA
-	}else{
-		cublasDcopy(rhs.size(), &(rhs[0]), 1, (double*)(&(r[0])), 2);
-		TEST_CUBLAS("cublasDcopy");//"NRSMat<std::complex<double> > complexify(const NRSMat<double> &)"
-	}
-#endif
-	return r;
-}
-*/
 /***************************************************************************//**
  * output operator
  * @param[in,out] s output stream

sparsemat.cc

@@ -334,7 +334,7 @@ return nonzero; //number of (in principle) nonzero elements
 
 
 template <class T>
-void SparseMat<T>::simplify(const double sparseepsilon)
+void SparseMat<T>::simplify(const typename LA_traits<T>::normtype sparseepsilon)
 {
 unsigned int n;
 if(!list) return;
@@ -431,7 +431,7 @@ void SparseMat<T>::addsafe(const SPMatindex n, const SPMatindex m, const T elem)
 #ifdef debug
 if(n<0||n>=nn||m<0||m>=mm) laerror("SparseMat out of range");
 #endif
-if(std::abs(elem)<=SPARSEEPSILON) return;
+if(std::abs(elem)<=LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) return;
 if(!count) {count=new int;  *count=1; list=NULL;} //blank new matrix
 else copyonwrite(); //makes also unsort
 add(n,m,elem);
@@ -482,7 +482,7 @@ register matel<T> *l=rhs.list;
 while(l)
 	{
 	if(rhs.symmetric || lower && l->row <=l->col || !lower && l->row >=l->col)
-	    if(std::abs(l->elem)>SPARSEEPSILON) 
+	    if(std::abs(l->elem)>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) 
 		add( l->row,l->col,sign=='+'?l->elem:- l->elem);
 	l=l->next;
 	}
@@ -501,14 +501,14 @@ register matel<T> *l=rhs.list;
 if(symmetrize)
 while(l)
         {
-        if(std::abs(l->elem)>SPARSEEPSILON) 
+        if(std::abs(l->elem)>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) 
 		{add( l->row,l->col,l->elem); if( l->row!=l->col) add( l->col,l->row,l->elem);}
         l=l->next;
         }
 else
 while(l)
 	{
-	if(std::abs(l->elem)>SPARSEEPSILON) 
+	if(std::abs(l->elem)>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) 
 		add( l->row,l->col,l->elem);
 	l=l->next;
 	}
@@ -528,14 +528,14 @@ register matel<T> *l=rhs.list;
 if(symmetrize)
 while(l)
         {
-        if(std::abs(l->elem)>SPARSEEPSILON) 
+        if(std::abs(l->elem)>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) 
 		{add( l->row,l->col,- l->elem); if( l->row!=l->col) add( l->col,l->row,- l->elem);}
         l=l->next;
         }
 else
 while(l)
         {
-        if(std::abs(l->elem)>SPARSEEPSILON) 
+        if(std::abs(l->elem)>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) 
 		add( l->row,l->col,- l->elem);
         l=l->next;
         }
@@ -559,7 +559,7 @@ SPMatindex i,j;
 for(i=0;i<nn;++i)
 	for(j=0; j<mm;++j)
 		{register T t(rhs(i,j));
-		if( std::abs(t)>SPARSEEPSILON)
+		if( std::abs(t)>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon())
 		 add(i,j,t);
 		}
 }
@@ -598,12 +598,12 @@ if(nn==mm) //square
 else //diagonal of A^TA, assuming it has been simplified (only one entry per position), will be used as preconditioner only anyway
 	while(l)
 		{
-		(*rr)[l->col] += l->elem*l->elem *(l->col!=l->row && symmetric?2.:1.);
+		(*rr)[l->col] += l->elem*l->elem *(T)(l->col!=l->row && symmetric?2:1);
 		l= l->next;
 		}
 if(divide)
 	{
-	for(unsigned int i=0; i<mm; ++i) if((*rr)[i]!=0.) r[i]/=(*rr)[i];
+	for(unsigned int i=0; i<mm; ++i) if((*rr)[i]!=(T)0) r[i]/=(*rr)[i];
 	delete rr;
 	}
 return divide?NULL:&r[0];
@@ -693,7 +693,7 @@ SparseMat<T> & SparseMat<T>::operator=(const T &a)
 if(!count ||nn<=0||mm<=0) laerror("assignment of scalar to undefined sparse matrix");
 if(nn!=mm) laerror("assignment of scalar to non-square sparse matrix");
 resize(nn,mm);//clear 
-if(std::abs(a)<=SPARSEEPSILON) return *this;
+if(std::abs(a)<=LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) return *this;
 SPMatindex i;
 for(i=0;i<nn;++i) add(i,i,a);
 return *this;
@@ -741,7 +741,7 @@ for(i=0;i<nn;++i)
         for(j=0; j<=i;++j)
 		{register T t;
                 if(
-			std::abs(t=rhs(i,j))>SPARSEEPSILON
+			std::abs(t=rhs(i,j))>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()
 			) add(i,j,t);
 		}
 }
@@ -774,7 +774,7 @@ matel<T> *l=list;
 while(l) //include the mirror picture of elements into the list
 	{
 	if(
-		std::abs(l->elem)>SPARSEEPSILON && 
+		std::abs(l->elem)>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon() && 
 	   l->row!=l->col) add(l->col,l->row,l->elem); //not OK for complex-hermitean
 	l=l->next;
 	}
@@ -812,12 +812,32 @@ while(l)
 return r;
 }
 
+template<>
+const float SparseMat<float>::dot(const NRMat<float> &rhs) const
+{
+float r=0;
+matel<float> *l=list;
+while(l)
+	{
+	r+= l->elem*rhs[l->row][l->col];
+	if(symmetric&&l->row!=l->col) r+=l->elem*rhs[l->col][l->row];
+	l=l->next;
+	}
+return r;
+}
+
 template<>
 void NRMat<std::complex<double> >::gemm(const std::complex<double>  &beta, const SparseMat<std::complex<double> > &a, const char transa, const NRMat<std::complex<double> > &b, const char transb, const std::complex<double>  &alpha)
 {
 laerror("not implemented yet");
 }
 
+template<>
+void NRMat<std::complex<float> >::gemm(const std::complex<float>  &beta, const SparseMat<std::complex<float> > &a, const char transa, const NRMat<std::complex<float> > &b, const char transb, const std::complex<float>  &alpha)
+{
+laerror("not implemented yet");
+}
+
 
 template<>
 void NRMat<double>::gemm(const double &beta, const SparseMat<double> &a, const char transa, const NRMat<double> &b, const char transb, const double &alpha)
@@ -867,6 +887,54 @@ else
 
 }
 
+template<>
+void NRMat<float>::gemm(const float &beta, const SparseMat<float> &a, const char transa, const NRMat<float> &b, const char transb, const float &alpha)
+{
+bool transpa = tolower(transa)!='n'; //not OK for complex
+bool transpb = tolower(transb)!='n'; //not OK for complex
+if(nn!=(int)(transpa?a.ncols():a.nrows()) | mm!= (transpb?b.nrows():b.ncols()) || (int)(transpa?a.nrows():a.ncols()) != (transpb?b.ncols():b.nrows())) laerror("incompatible sizes in gemm");
+copyonwrite();
+if(beta!=(float)0) (*this) *= beta; 
+else memset(v,0,nn*mm*sizeof(float));
+
+matel<float> *l=a.getlist();
+if(alpha==(float)0 || !l) return;
+
+//raw pointers to the full matrices
+const float *bp= b;
+float *p= *this;
+
+int ldb=(transpb?b.ncols():1);
+int bstep=(transpb?1:b.ncols());
+int len=(transpb?b.nrows():b.ncols());
+
+if(a.issymmetric())
+	{
+	while(l)
+	        {
+		cblas_saxpy(len, alpha*l->elem , bp+l->row*bstep, ldb, p+l->col*mm, 1);
+	        if(l->row!=l->col) cblas_saxpy(len, alpha*l->elem , bp+l->col*bstep, ldb, p+l->row*mm, 1);
+	        l=l->next;
+	        }
+	}
+else
+	{
+	if(transpa)
+	while(l)
+	        {
+		cblas_saxpy(len, alpha*l->elem , bp+l->row*bstep, ldb, p+l->col*mm, 1);
+	        l=l->next;
+	        }
+	else
+	while(l)
+	        {
+		cblas_saxpy(len, alpha*l->elem , bp+l->col*bstep, ldb, p+l->row*mm, 1);
+	        l=l->next;
+	        }
+	}
+
+}
+
 
 
 template<class T>
@@ -1036,7 +1104,7 @@ if(transp)
 while(l)
         {
 	register T t=alpha*l->elem;
-	if(std::abs(t)>SPARSEEPSILON) 
+	if(std::abs(t)>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) 
 				add( l->col,l->row,t);
         l=l->next;
         }
@@ -1044,7 +1112,7 @@ else
 while(l)
 	{
 	register T t=alpha*l->elem;
-	if(std::abs(t)>SPARSEEPSILON) 
+	if(std::abs(t)>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) 
 				add( l->row,l->col,t);
 	l=l->next;
 	}
@@ -1329,7 +1397,7 @@ incsize(rhs.nrows(), rhs.ncols());
 T tmp;
 for(SPMatindex i=0; i<(SPMatindex)rhs.nrows(); ++i)
 	for(SPMatindex j=0; j<(SPMatindex)rhs.ncols(); ++j)
-		if(std::abs(tmp=rhs(i,j))>SPARSEEPSILON) add(n0+i,m0+j,tmp);
+		if(std::abs(tmp=rhs(i,j))>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) add(n0+i,m0+j,tmp);
 return *this;
 }
 
@@ -1345,7 +1413,7 @@ T tmp;
 incsize(rhs.nrows(), rhs.ncols());
 for(SPMatindex i=0; i<(SPMatindex)rhs.nrows(); ++i)
         for(SPMatindex j=0; j<=i; ++j) //symmetric storage: one triangle only
-                if(std::abs(tmp=rhs(i,j))>SPARSEEPSILON) add(n0+i,m0+j,tmp);
+                if(std::abs(tmp=rhs(i,j))>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) add(n0+i,m0+j,tmp);
 return *this;
 }
 
@@ -1364,7 +1432,7 @@ incsize(rhs.nrows(), rhs.ncols());
 register matel<T> *l=rhs.list;
 while(l)
 	{
-	if(std::abs(l->elem)>SPARSEEPSILON) 
+	if(std::abs(l->elem)>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) 
 		add(n0+l->row,m0+l->col,l->elem);
 	l=l->next;
 	}
@@ -1429,8 +1497,12 @@ INSTANTIZE(std::complex<double>) //some functions are not OK for hermitean matri
 //// forced instantization in the corresponding object file
 template class SparseMat<double>;
 template class SparseMat<std::complex<double> >;
+template class SparseMat<float>;
+template class SparseMat<std::complex<float> >;
 template class SemiSparseMat<double>;
 template class SemiSparseMat<std::complex<double> >;
+template class SemiSparseMat<float>;
+template class SemiSparseMat<std::complex<float> >;
 
 
 #define INSTANTIZE(T) \
@@ -1443,6 +1515,8 @@ template void NRVec<T>::gemv(const T beta, const SparseMat<T> &a, const char tra
 
 INSTANTIZE(double)
 INSTANTIZE(std::complex<double>)
+INSTANTIZE(float)
+INSTANTIZE(std::complex<float>)
 
 
 }//namespace

sparsemat.h

@@ -80,6 +80,7 @@ public:
 	inline int getcount() const {return count?*count:0;}
 	explicit SparseMat(const NRMat<T> &rhs); //construct from a dense one
 	explicit SparseMat(const NRSMat<T> &rhs); //construct from a dense symmetric one
+	explicit SparseMat(const CSRMat<T> &rhs); //full storage, not symmetric
 	SparseMat & operator=(const SparseMat &rhs);
 	SparseMat & operator=(const T &a);          //assign a to diagonal
 	void identity() {*this = (T)1;};
@@ -141,7 +142,7 @@ public:
 	unsigned int length() const;
 	void copyonwrite(bool detachonly=false, bool deep=true);
 	void clear() {bool sym=symmetric; resize(nn,mm); symmetric=sym;} //zero matrix of the same dimensions and symmetry
-	void simplify(const double sparseepsilon=SPARSEEPSILON);
+	void simplify(const typename LA_traits<T>::normtype sparseepsilon=LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon());
 	const T trace() const;
 	const typename LA_traits<T>::normtype norm(const T scalar=(T)0) const; //is const only mathematically, not in internal implementation - we have to simplify first
 	unsigned int sort(int type) const;

sparsesmat.cc

@@ -239,7 +239,7 @@ for(SPMatindex i=0; i<nn; ++i)
 		}
 if(divide)
 	{
-	for(unsigned int i=0; i<nn; ++i) if((*rr)[i]!=0.) r[i]/=(*rr)[i];
+	for(unsigned int i=0; i<nn; ++i) if((*rr)[i]!=(T)0) r[i]/=(*rr)[i];
 	delete rr;
 	}
 return divide?NULL:&r[0];
@@ -352,12 +352,18 @@ INSTANTIZE(complex<double>)
 //// forced instantization of functions in the header in the corresponding object file
 template class SparseSMat<double>;
 template class SparseSMat<std::complex<double> >;
+template class SparseSMat<float>;
+template class SparseSMat<std::complex<float> >;
 template SparseSMat<double> NRMat<double>::operator*(const SparseSMat<double> &rhs) const;
+template SparseSMat<float> NRMat<float>::operator*(const SparseSMat<float> &rhs) const;
 template SparseSMat<std::complex<double> > NRMat<std::complex<double> >::operator*(const SparseSMat<std::complex<double> > &rhs) const;
+template SparseSMat<std::complex<float> > NRMat<std::complex<float> >::operator*(const SparseSMat<std::complex<float> > &rhs) const;
 
 /*activate this if needed
 template void SparseSMat<NRMat<double> >::put(int fd, bool dimen, bool transp, bool orcaformat) const;
+template void SparseSMat<NRMat<float> >::put(int fd, bool dimen, bool transp, bool orcaformat) const;
 template void SparseSMat<NRMat<double> >::get(int fd, bool dimen, bool transp, bool orcaformat);
+template void SparseSMat<NRMat<float> >::get(int fd, bool dimen, bool transp, bool orcaformat);
 */
 
 

sparsesmat.h

@@ -200,7 +200,7 @@ count(new int(1))
 v= new std::map<SPMatindex,T> * [nn];
 memset(v,0,nn*sizeof(std::map<SPMatindex,T> *));
 int i,j;
-for(i=0; i<nn; ++i) for(j=0; j<=i; ++j) if(std::abs(rhs(i,j))>SPARSEEPSILON) (*this).add(i,j,rhs(i,j),true);
+for(i=0; i<nn; ++i) for(j=0; j<=i; ++j) if(std::abs(rhs(i,j))>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) (*this).add(i,j,rhs(i,j),true);
 }
 
 //from SparseMat: duplicates summed; a symmetric SparseMat (one triangle stored, complex symmetric) is unfolded without conjugation
@@ -227,7 +227,7 @@ count(new int(1))
 v= new std::map<SPMatindex,T> * [nn];
 memset(v,0,nn*sizeof(std::map<SPMatindex,T> *));
 int i,j;
-for(i=0; i<nn; ++i) for(j=0; j<mm; ++j) if(std::abs(rhs(i,j))>SPARSEEPSILON) (*this).add(i,j,rhs(i,j),false);
+for(i=0; i<nn; ++i) for(j=0; j<mm; ++j) if(std::abs(rhs(i,j))>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) (*this).add(i,j,rhs(i,j),false);
 }
 
 
@@ -437,7 +437,7 @@ for(SPMatindex i=0; i<nn; ++i) if(v[i])
 	std::list<SPMatindex> l;
 	typename std::map<SPMatindex,T>::iterator p;
 	for(p=v[i]->begin(); p!=v[i]->end(); ++p) 
-	if(std::abs(p->second) < SPARSEEPSILON) l.push_front(p->first); else ++count;
+	if(std::abs(p->second) < LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) l.push_front(p->first); else ++count;
 	typename std::list<SPMatindex>::iterator q;	
 	for(q=l.begin(); q!=l.end(); ++q) v[i]->erase(*q);	
 	if(v[i]->size() == 0) {delete v[i]; v[i]=NULL;}
@@ -548,7 +548,7 @@ for(SPMatindex i=0; i<nn; ++i)
 //std::cout <<"Permuted upper triangle matrix\n"<<r;
 //SparseSMat<T> r0(r);r.copyonwrite();
 
-//perform complex, positive semidefinite Cholesky with thresholding by SPARSEEPSILON
+//perform complex, positive semidefinite Cholesky with thresholding by LA_precision<>::sparse_epsilon()
 SPMatindex i,j,k;
 int rank=0;
 for(k=0; k<nn; ++k)
@@ -557,7 +557,7 @@ for(k=0; k<nn; ++k)
 	typename std::map<SPMatindex,T>::iterator p;
 	p= r.v[k]->find(k);
 	if(p==r.v[k]->end()) continue; //must not break due to the a priori  pivoting
-	if(LA_traits<T>::realpart(p->second) < CHOLESKYEPSILON) continue; //must not break due to the a priori  pivoting
+	if(LA_traits<T>::realpart(p->second) < LA_precision<typename LA_traits<T>::normtype>::cholesky_epsilon()) continue; //must not break due to the a priori  pivoting
 	++rank;
 	typename LA_traits<T>::normtype tmp = std::sqrt(LA_traits<T>::realpart(p->second));
 	p->second = tmp;
@@ -572,7 +572,7 @@ for(k=0; k<nn; ++k)
 		{
 		T akj = LA_traits<T>::conjugate(linek[j]);
 		NRVec<int> linek_used(0,nn);
-		if(std::abs(akj)>SPARSEEPSILON) 
+		if(std::abs(akj)>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) 
 			{
 			for(p=r.v[j]->begin(); p!=r.v[j]->end(); ++p)
 				{
@@ -581,7 +581,7 @@ for(k=0; k<nn; ++k)
 				}	
 			//subtract also elements nonzero in line k but non-existent in line j
 			for(i=j; i<nn; ++i) 
-			    if(!linek_used[i] && std::abs(linek[i]) > SPARSEEPSILON)
+			    if(!linek_used[i] && std::abs(linek[i]) > LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon())
 				{
 				(*r.v[j])[i] = -akj * linek[i];
 				}
@@ -611,7 +611,7 @@ for(k=0; k<nn; ++k)
 	typename std::map<SPMatindex,T> *vnew = new typename std::map<SPMatindex,T>;
 	for(p=r.v[k]->begin(); p!=r.v[k]->end(); ++p)
 		{
-        	if(std::abs(p->second) > SPARSEEPSILON) (*vnew)[pivot[p->first]] = p->second;
+        	if(std::abs(p->second) > LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) (*vnew)[pivot[p->first]] = p->second;
 		}
 	delete r.v[k];
 	r.v[k]=vnew;
@@ -636,7 +636,7 @@ for(SPMatindex i=0; i<lhs.size(); ++i)
 		{
 		T x=lhs[i]*(conjugate?LA_traits<T>::conjugate(rhs[j]):rhs[j])*scale;
 		if(i==j) x+=LA_traits<T>::conjugate(x); //diagonal of M + M^\dagger, add() mirrors only off-diagonal elements
-		if(std::abs(x)>SPARSEEPSILON) r.add(i,j,x);
+		if(std::abs(x)>LA_precision<typename LA_traits<T>::normtype>::sparse_epsilon()) r.add(i,j,x);
 		}
 	}
 return r;

strassen.cc

@@ -19,32 +19,159 @@
 
 #include "mat.h"
 
-#ifndef NO_STRASSEN
 namespace LA {
-/*Strassen algorithm*/
+/*Strassen algorithm in the Winograd variant (7 products, 15 additions), recursion on strided quadrants of the original matrices,
-// called routine is fortran-compatible
+leaf products by BLAS gemm
-extern "C" void fmm(const char c_transa,const char c_transb,const int m,const int n,const int k,const double alpha,
+V. Strassen, Gaussian elimination is not optimal, Numer. Math. 13 (1969) 354-356
-                const double *a,const int lda,const double *b,const int ldb,const double beta,double *c,const int ldc,
+S. Winograd, On multiplication of 2x2 matrices, Linear Algebra Appl. 4 (1971) 381-388
-                double *d_aux,int i_naux);
+schedule with 2 temporaries per level: B. Boyer, J.-G. Dumas, C. Pernet, W. Zhou, Memory efficient scheduling of
-extern "C" void strassen_cutoff(int c, int c1, int c2, int c3);
+	Strassen-Winograd's matrix multiplication algorithm, Proc. ISSAC 2009, 55-62
+odd dimensions by dynamic peeling: S. Huss-Lederman, E. M. Jacobson, J. R. Johnson, A. Tsao, T. Turnbull,
+	Implementation of Strassen's algorithm for matrix multiplication, Proc. Supercomputing '96
+*/
 
-template<>
+//device-dependent primitives: leaf gemm, quadrant addition, workspace (GPU version: cublas?gemm, cublas?geam, gpualloc/gpufree)
-void NRMat<double>::s_cutoff(const int c, const int c1, const int c2, const int c3) const
+static inline CBLAS_TRANSPOSE s_trans(const char t) {return t=='n'?CblasNoTrans:(t=='c'?CblasConjTrans:CblasTrans);}
-{ strassen_cutoff(c,c1,c2,c3);}
 
-template<>
+static inline void s_gemm(const char ta, const char tb, const int m, const int n, const int k, const double alpha, const double *a, const int lda, const double *b, const int ldb, const double beta, double *c, const int ldc)
-void NRMat<double>::strassen(const double beta, const NRMat<double> &a, const char transa, const NRMat<double> &b, const char transb, const double alpha)
+{cblas_dgemm(CblasRowMajor,s_trans(ta),s_trans(tb),m,n,k,alpha,a,lda,b,ldb,beta,c,ldc);}
+static inline void s_gemm(const char ta, const char tb, const int m, const int n, const int k, const float alpha, const float *a, const int lda, const float *b, const int ldb, const float beta, float *c, const int ldc)
+{cblas_sgemm(CblasRowMajor,s_trans(ta),s_trans(tb),m,n,k,alpha,a,lda,b,ldb,beta,c,ldc);}
+static inline void s_gemm(const char ta, const char tb, const int m, const int n, const int k, const std::complex<double> alpha, const std::complex<double> *a, const int lda, const std::complex<double> *b, const int ldb, const std::complex<double> beta, std::complex<double> *c, const int ldc)
+{cblas_zgemm(CblasRowMajor,s_trans(ta),s_trans(tb),m,n,k,&alpha,a,lda,b,ldb,&beta,c,ldc);}
+static inline void s_gemm(const char ta, const char tb, const int m, const int n, const int k, const std::complex<float> alpha, const std::complex<float> *a, const int lda, const std::complex<float> *b, const int ldb, const std::complex<float> beta, std::complex<float> *c, const int ldc)
+{cblas_cgemm(CblasRowMajor,s_trans(ta),s_trans(tb),m,n,k,&alpha,a,lda,b,ldb,&beta,c,ldc);}
+
+//z = x+y or x-y, r x c blocks with leading dimensions; z may coincide with x or y (one pass, memory bound)
+template<typename T>
+static void s_add(const int r, const int c, T *z, const int ldz, const T *x, const int ldx, const T *y, const int ldy, const bool minus)
 {
-int l(transa=='n'?a.nn:a.mm);
+for(int i=0; i<r; ++i)
-int k(transa=='n'?a.mm:a.nn);
+	{
-int kk(transb=='n'?b.nn:b.mm);
+	T *zz=z+(size_t)i*ldz; const T *xx=x+(size_t)i*ldx; const T *yy=y+(size_t)i*ldy;
-int ll(transb=='n'?b.mm:b.nn);
+	if(minus) for(int j=0; j<c; ++j) zz[j]=xx[j]-yy[j];
-
+	else for(int j=0; j<c; ++j) zz[j]=xx[j]+yy[j];
-if(l!=nn|| ll!=mm||k!=kk) laerror("incompatible (or undefined size) matrices in strassen");
-
-copyonwrite();
-//swap transpositions and order of matrices
-fmm(transb,transa,mm,nn,k,alpha,b,b.mm, a, a.mm, beta,*this, mm,NULL,0);
 	}
-}//namespace
+}
+
+template<typename T> static T *s_alloc(const size_t n) {return n? new T[n] : NULL;}
+template<typename T> static void s_free(T *p) {if(p) delete[] p;}
+
+//operand op(M) of the product, op = identity ('n'), transposition ('t') or conjugate transposition ('c'),
+//M stored row-major at p with leading dimension ld; the recursion is formulated in op() space and never transposes data:
+//- at(i,j) is the address of element (i,j) of op(M), i.e. M(i,j) for 'n' and M(j,i) for 't','c';
+//  a block of op(M) starting at (i,j) is then the block of M starting at at(i,j) with the same ld and flag
+//  (quadrant (op M)_{IJ} = op(M_{JI}), the stored quadrant being swapped for 't','c' and conjugated by gemm for 'c')
+//- sums of quadrants are formed elementwise in the stored layout (S stored = M_{JI} + M_{LK} for 't','c'), which is correct
+//  because op(X+Y)=op(X)+op(Y), so the temporaries keep the flag of their operand and gemm applies op() at the leaves only
+template<typename T>
+struct s_op
+	{
+	const T *p; int ld; char t;
+	const T *at(const int i, const int j) const {return t=='n' ? p+(size_t)i*ld+j : p+(size_t)j*ld+i;}
+	};
+
+static inline bool s_split(const int m, const int n, const int k, const int cutoff) {return m>cutoff && n>cutoff && k>cutoff;}
+
+//workspace needed for all levels below (m,n,k)
+static size_t s_work(const int m, const int n, const int k, const int cutoff)
+{
+if(!s_split(m,n,k,cutoff)) return 0;
+size_t mh=m/2, nh=n/2, kh=k/2;
+return (mh*kh>mh*nh ? mh*kh : mh*nh) + kh*nh + s_work(mh,nh,kh,cutoff);
+}
+
+//C(m x n, ldc) = alpha op(A) op(B)
+template<typename T>
+static void s_rec(const int m, const int n, const int k, const T alpha, const s_op<T> &A, const s_op<T> &B, T *C, const int ldc, T *ws, const int cutoff)
+{
+if(!s_split(m,n,k,cutoff)) {s_gemm(A.t,B.t,m,n,k,alpha,A.p,A.ld,B.p,B.ld,(T)0,C,ldc); return;}
+const int mh=m/2, nh=n/2, kh=k/2;
+//stored dimensions of the quadrants of A and B
+const int ar= A.t=='n'?mh:kh, ac= A.t=='n'?kh:mh;
+const int br= B.t=='n'?kh:nh, bc= B.t=='n'?nh:kh;
+const T *A11=A.at(0,0), *A12=A.at(0,kh), *A21=A.at(mh,0), *A22=A.at(mh,kh);
+const T *B11=B.at(0,0), *B12=B.at(0,nh), *B21=B.at(kh,0), *B22=B.at(kh,nh);
+T *C11=C, *C12=C+nh, *C21=C+(size_t)mh*ldc, *C22=C21+nh;
+const size_t xs= (size_t)mh*kh>(size_t)mh*nh ? (size_t)mh*kh : (size_t)mh*nh;
+T *X=ws, *Y=ws+xs, *wn=Y+(size_t)kh*nh;
+const s_op<T> sx={X,ac,A.t}, sy={Y,bc,B.t};	//X, Y hold sums of quadrants of A, B in their stored layout, contiguous (ld = stored columns), same flag
+const s_op<T> a11={A11,A.ld,A.t}, a12={A12,A.ld,A.t}, a22={A22,A.ld,A.t};
+const s_op<T> b11={B11,B.ld,B.t}, b21={B21,B.ld,B.t}, b22={B22,B.ld,B.t};
+s_add(ar,ac,X,ac,A11,A.ld,A21,A.ld,true);	//S3=A11-A21
+s_add(br,bc,Y,bc,B22,B.ld,B12,B.ld,true);	//T3=B22-B12
+s_rec(mh,nh,kh,alpha,sx,sy,C21,ldc,wn,cutoff);	//P7
+s_add(ar,ac,X,ac,A21,A.ld,A22,A.ld,false);	//S1=A21+A22
+s_add(br,bc,Y,bc,B12,B.ld,B11,B.ld,true);	//T1=B12-B11
+s_rec(mh,nh,kh,alpha,sx,sy,C22,ldc,wn,cutoff);	//P5
+s_add(ar,ac,X,ac,X,ac,A11,A.ld,true);		//S2=S1-A11
+s_add(br,bc,Y,bc,B22,B.ld,Y,bc,true);		//T2=B22-T1
+s_rec(mh,nh,kh,alpha,sx,sy,C12,ldc,wn,cutoff);	//P6
+s_add(ar,ac,X,ac,A12,A.ld,X,ac,true);		//S4=A12-S2
+s_rec(mh,nh,kh,alpha,sx,b22,C11,ldc,wn,cutoff);	//P3
+s_rec(mh,nh,kh,alpha,a11,b11,X,nh,wn,cutoff);	//P1 in X
+s_add(mh,nh,C12,ldc,X,nh,C12,ldc,false);	//U2=P1+P6
+s_add(mh,nh,C21,ldc,C12,ldc,C21,ldc,false);	//U3=U2+P7
+s_add(mh,nh,C12,ldc,C12,ldc,C22,ldc,false);	//U4=U2+P5
+s_add(mh,nh,C22,ldc,C21,ldc,C22,ldc,false);	//C22=U3+P5
+s_add(mh,nh,C12,ldc,C12,ldc,C11,ldc,false);	//C12=U4+P3
+s_add(br,bc,Y,bc,Y,bc,B21,B.ld,true);		//T4=T2-B21
+s_rec(mh,nh,kh,alpha,a22,sy,C11,ldc,wn,cutoff);	//P4
+s_add(mh,nh,C21,ldc,C21,ldc,C11,ldc,true);	//C21=U3-P4
+s_rec(mh,nh,kh,alpha,a12,b21,C11,ldc,wn,cutoff);	//P2
+s_add(mh,nh,C11,ldc,X,nh,C11,ldc,false);	//C11=P1+P2
+//odd dimensions
+const int me=2*mh, ne=2*nh, ke=2*kh;
+if(k>ke) s_gemm(A.t,B.t,me,ne,1,alpha,A.at(0,ke),A.ld,B.at(ke,0),B.ld,(T)1,C,ldc);
+if(n>ne) s_gemm(A.t,B.t,me,1,k,alpha,A.p,A.ld,B.at(0,ne),B.ld,(T)0,C+ne,ldc);
+if(m>me) s_gemm(A.t,B.t,1,n,k,alpha,A.at(me,0),A.ld,B.p,B.ld,(T)0,C+(size_t)me*ldc,ldc);
+}
+
+//C = beta C + alpha op(A) op(B), row-major m x n
+template<typename T>
+static void s_top(const int m, const int n, const int k, const T alpha, const T *a, const int lda, const char ta, const T *b, const int ldb, const char tb, const T beta, T *c, const int cutoff)
+{
+const s_op<T> A={a,lda,ta}, B={b,ldb,tb};
+if(!s_split(m,n,k,cutoff)) {s_gemm(ta,tb,m,n,k,alpha,a,lda,b,ldb,beta,c,n); return;}
+const size_t w=s_work(m,n,k,cutoff);
+const bool acc= beta!=(T)0;
+T *ws=s_alloc<T>(w+(acc?(size_t)m*n:0));
+T *cc= acc? ws+w : c;
+s_rec(m,n,k,alpha,A,B,cc,n,ws,cutoff);
+if(acc) for(size_t i=0; i<(size_t)m*n; ++i) c[i]=beta*c[i]+cc[i];
+s_free(ws);
+}
+
+static inline char s_flag(const char t, const bool cmplx) {char tt=tolower(t); return tt=='n'?'n':(tt=='c'&&cmplx?'c':'t');}
+
+template<typename T>
+static void s_strassen(NRMat<T> &c, const T &beta, const NRMat<T> &a, const char transa, const NRMat<T> &b, const char transb, const T &alpha, const int cutoff)
+{
+const char ta=s_flag(transa,LA_traits<T>::is_complex()), tb=s_flag(transb,LA_traits<T>::is_complex());
+int k(ta=='n'?a.ncols():a.nrows());
+#ifdef DEBUG
+int l(ta=='n'?a.nrows():a.ncols());
+int kk(tb=='n'?b.nrows():b.ncols());
+int ll(tb=='n'?b.ncols():b.nrows());
+if (l!=c.nrows() || ll!=c.ncols() || k!=kk) laerror("incompatible matrices in NRMat<T>::strassen(...)");
+if(b.ncols() <=0 || c.ncols()<=0) laerror("illegal matrix dimension in strassen");
 #endif
+SAME_LOC3(c, a, b);
+NOT_GPU(c);
+if(cutoff<1) laerror("illegal cutoff in strassen");
+if(alpha==(T)0 && beta==(T)1) return;
+c.copyonwrite();
+s_top(c.nrows(),c.ncols(),k,alpha,(const T *)a,a.ncols(),ta,(const T *)b,b.ncols(),tb,beta,(T *)c,cutoff);
+}
+
+#define INSTANTIZE(T) \
+template<> \
+void NRMat<T>::strassen(const T &beta, const NRMat<T> &a, const char transa, const NRMat<T> &b, const char transb, const T &alpha, const int cutoff) \
+{s_strassen(*this,beta,a,transa,b,transb,alpha,cutoff);}
+
+INSTANTIZE(double)
+INSTANTIZE(float)
+INSTANTIZE(std::complex<double>)
+INSTANTIZE(std::complex<float>)
+
+}//namespace

t.cc

@@ -324,16 +324,11 @@ cout <<" big1*big2 "<<big3[0][0]<<" time "<<clock()/((double) (CLOCKS_PER_SEC))-
 
 */
 
-#ifndef NO_STRASSEN
 if(0)
 {
 NRMat<double> atest, btest,ctest;
-{
+int cc;
-int cc,c1,c2,c3;
+cin >>cc;
-cin >>cc>>c1>>c2>>c3;
-
-atest.s_cutoff(cc,c1,c2,c3);
-}
 cin>>atest;
 cin>>btest;
 
@@ -342,7 +337,7 @@ dtest.gemm(0., atest, 't', btest, 'n', 1.);
 cout << dtest;
 
 NRMat<double> etest(atest.nrows(),btest.ncols());
-etest.strassen(0., atest, 't', btest, 'n', 1.);
+etest.strassen(0., atest, 't', btest, 'n', 1., cc);
 cout << etest;
 }
 
@@ -363,13 +358,11 @@ cout <<" classical big1*big2 "<<big3[0][0]<<" time "<<clock()/((double) (CLOCKS_
 
 for (int c=64; c<=512;c+=64)
 	{
-	big4.s_cutoff(c,c,c,c);
 	t=clock()/((double) (CLOCKS_PER_SEC));
-	big4.strassen(0., big1, 'n', big2, 'n', 1.);
+	big4.strassen(0., big1, 'n', big2, 'n', 1., c);
 	cout <<"cutoff "<<c<<" big1*big2 "<<big4[0][0]<<" time "<<clock()/((double) (CLOCKS_PER_SEC))-t <<"\n";
 	}
 }
-#endif
 
 if(0)
 {
@@ -1887,7 +1880,6 @@ c.moveto(gpu1);
 cout << "Error = "<<(c-cgpu).norm()<<endl;
 }
 
-/*
 if(0)
 {
 CSRMat<double> h0;
@@ -1910,7 +1902,6 @@ NRMat<complex<double> >r2 = exp(h2);
 cout <<"error = "<<(r2-NRMat<complex<double> >(r)).norm()<<endl;
 }
 }
-*/
 
 
 if(0)

tensor.cc

@@ -1187,6 +1187,16 @@ std::cout<<"cblas_dgemm CPU time " << (clock()-t0)/(double)CLOCKS_PER_SEC<<std::
 #endif
 }
 
+template<>
+void auxmatmult<float>(int nn, int mm, int kk, float *r, const float *a, const float *b, float alpha, float beta, bool conjugate)
+{
+clock_t t0=clock();
+cblas_sgemm(CblasRowMajor, CblasNoTrans, CblasTrans, nn, mm, kk, alpha, a, kk, b, kk, beta, r, mm);
+#ifdef TIMING
+std::cout<<"cblas_sgemm CPU time " << (clock()-t0)/(double)CLOCKS_PER_SEC<<std::endl;
+#endif
+}
+
 template<>
 void auxmatmult<std::complex<double> >(int nn, int mm, int kk, std::complex<double> *r, const std::complex<double> *a, const std::complex<double> *b,  std::complex<double> alpha,  std::complex<double> beta, bool conjugate)
 {
@@ -1197,6 +1207,16 @@ std::cout<<"cblas_zgemm CPU time " << (clock()-t0)/(double)CLOCKS_PER_SEC<<std::
 #endif
 }
 
+template<>
+void auxmatmult<std::complex<float> >(int nn, int mm, int kk, std::complex<float> *r, const std::complex<float> *a, const std::complex<float> *b,  std::complex<float> alpha,  std::complex<float> beta, bool conjugate)
+{
+clock_t t0=clock();
+cblas_cgemm(CblasRowMajor, CblasNoTrans, (conjugate?CblasConjTrans:CblasTrans), nn, mm, kk, &alpha, a, kk, b, kk, &beta, r, mm);
+#ifdef TIMING
+std::cout<<"cblas_cgemm CPU time " << (clock()-t0)/(double)CLOCKS_PER_SEC<<std::endl;
+#endif
+}
+
 
 
 template<typename T>
@@ -1809,7 +1829,7 @@ for(int i=0; i<r; ++i)
 		umnew=u.submatrix(0,umnr-1,0,preserve-1);
 		}
 	else umnew=u;
-	ret[r-i-1]=vt.transpose(true);
+	ret[r-i-1]=vt.transpose(false);
 	umnew.diagmultr(w);
 	//rebuild tensor of the preserved shape from matrix
 	ushape.copyonwrite();
@@ -1825,7 +1845,7 @@ if(!is_flat()) laerror("this should not happen");
 if(inverseorder)
 	{
 	NRPerm<int> p(rank()); p.identity();
-	names=names_saved.permuted(p.reverse());
+	if(names_saved.size()) names=names_saved.permuted(p.reverse());
 	}
 else 
 	names=names_saved;
@@ -1869,7 +1889,7 @@ for(int i=rank()-1; i>=0; --i)
 if(inverseorder)
         {
         NRPerm<int> p(rank()); p.identity();
-        r.names=names.permuted(p.reverse());
+        if(is_named()) r.names=names.permuted(p.reverse());
         }
 else
         r.names=names;
@@ -2646,6 +2666,12 @@ template std::ostream & operator<<(std::ostream &s, const Tensor<double> &x);
 template std::ostream & operator<<(std::ostream &s, const Tensor<std::complex<double> > &x);
 template std::istream & operator>>(std::istream &s, Tensor<double> &x);
 template std::istream & operator>>(std::istream &s, Tensor<std::complex<double> > &x);
+template class Tensor<float>;
+template class Tensor<std::complex<float> >;
+template std::ostream & operator<<(std::ostream &s, const Tensor<float> &x);
+template std::ostream & operator<<(std::ostream &s, const Tensor<std::complex<float> > &x);
+template std::istream & operator>>(std::istream &s, Tensor<float> &x);
+template std::istream & operator>>(std::istream &s, Tensor<std::complex<float> > &x);
 
 
 }//namespace

tensor.h

@@ -446,7 +446,7 @@ public:
 	Tensor merge_indices(const INDEXLIST &il, int symmetry=0) const;	//opposite to flatten (merging with optional symmetrization/antisymmetrization and compression)
 	Tensor merge_indices(const NRVec<INDEXNAME> &nl, int symmetry=0) const {return merge_indices(findindexlist(nl),symmetry);};
 
-	NRVec<NRMat<T> > Tucker(typename LA_traits<T>::normtype thr=1e-12, bool inverseorder=false); //HOSVD-Tucker decomposition, return core tensor in *this, flattened 
+	NRVec<NRMat<T> > Tucker(typename LA_traits<T>::normtype thr=LA_precision<typename LA_traits<T>::normtype>::svd_threshold(), bool inverseorder=false); //HOSVD-Tucker decomposition, return core tensor in *this, flattened 
 	Tensor inverseTucker(const NRVec<NRMat<T> > &x, bool inverseorder=false) const; //rebuild the original tensor from Tucker
 	Tensor linear_transform(const NRVec<NRMat<T> > &x) const; //linear transform by a different matrix per each index group, preserving group symmetries
 	Tensor linear_transform(const int g, const Tensor<T> &x) const; //linear transform a single group, preserve its position and symmetry, x must be rank-2 flat tensor (this is useful for raising/lowering indices)

testoptimize.cc

@@ -936,6 +936,77 @@ check("NRSMat::storesubmatrix(selection) copy-on-write and size check",unchanged
 }
 
 
+//single precision: every method with the float defaults (opt_precision<float>) on float test problems
+template<class O, class H>
+static float runf(O &o, const OptProblem<float> *p, NRVec<float> &x, H &h, bool *converged)
+{
+h.my_verbose=0;
+x.copyonwrite();
+optresult<float> r=o.minimize(p,x,&h,NULL);
+*converged=r.converged;
+return r.f;
+}
+
+static NRVec<float> startpointf(const int n, const int seed)
+{
+NRVec<double> xd=startpoint(n,seed);
+NRVec<float> x(n);
+for(int i=0; i<n; ++i) x[i]=xd[i];
+return x;
+}
+
+//the same method in double and in float from the same start, the float run with its default thresholds
+template<template<class> class O, template<class> class H>
+static void compare_float(const char *name, const OptProblem<double> *pd, const OptProblem<float> *pf, const int n, const int seed, const int maxstep, void (*modify)(H<double> &, H<float> &)=NULL)
+{
+NRVec<double> xd=startpoint(n,seed); NRVec<float> xf(n); for(int i=0; i<n; ++i) xf[i]=xd[i];
+H<double> hd; H<float> hf; hd.my_maxstep=hf.my_maxstep=maxstep; if(modify) modify(hd,hf);
+O<double> od; O<float> of;
+bool convd, convf;
+hd.my_verbose=0; xd.copyonwrite(); optresult<double> rd=od.minimize(pd,xd,&hd,NULL); convd=rd.converged;
+float ff=runf(of,pf,xf,hf,&convf);
+string s=string("float ")+name+" agrees with double";
+check(s.c_str(),convd && convf && abs(ff-rd.f)<1e-5*max(1.,abs(rd.f)),ff-rd.f);
+}
+static void dfp(BFGS_hyperparameters<double> &hd, BFGS_hyperparameters<float> &hf) {hd.my_update=hf.my_update=QN_DFP; hd.my_c2=hf.my_c2=0.1;}
+static void sr1(BFGS_hyperparameters<double> &hd, BFGS_hyperparameters<float> &hf) {hd.my_update=hf.my_update=QN_SR1;}
+
+static void test_float()
+{
+cout <<"--- single precision (float)\n";
+testproblem<float> tp; //f=0 at x=0, used for the derivative-free methods; its large constant term n(n-1)/2 cancels in f, so f is not
+		      //resolved below about 3e-6 in float and gradient methods cannot reach g_thr there: they use testproblem_vqe
+testproblem_vqe<double> vd; testproblem_vqe<float> vf;
+compare_float<BFGS_Optimizer,BFGS_hyperparameters>("BFGS on testproblem_vqe",&vd,&vf,8,3,5000);
+compare_float<BFGS_Optimizer,BFGS_hyperparameters>("DFP on testproblem_vqe",&vd,&vf,8,3,5000,&dfp);
+compare_float<BFGS_Optimizer,BFGS_hyperparameters>("SR1 on testproblem_vqe",&vd,&vf,8,3,5000,&sr1);
+compare_float<LBFGS_Optimizer,LBFGS_hyperparameters>("L-BFGS on testproblem_vqe",&vd,&vf,8,3,5000);
+compare_float<CG_Optimizer,CG_hyperparameters>("CG on testproblem_vqe",&vd,&vf,8,3,5000);
+compare_float<GDIIS_Optimizer,GDIIS_hyperparameters>("GDIIS on testproblem_vqe",&vd,&vf,8,3,5000);
+{NelderMead_hyperparameters<float> h; h.my_maxstep=100000; NelderMead_Optimizer<float> o; NRVec<float> x=startpointf(4,1); bool conv; float f=runf(o,&tp,x,h,&conv); check("float Nelder-Mead converges on testproblem n=4 (default f_thr, x_thr)",conv && f<1e-5,f);}
+{Powell_hyperparameters<float> h; Powell_Optimizer<float> o; NRVec<float> x=startpointf(10,1); bool conv; float f=runf(o,&tp,x,h,&conv); check("float Powell converges on testproblem n=10 (default f_thr, x_thr)",conv && f<1e-5,f);}
+{Cobyla_hyperparameters<float> h; Cobyla_Optimizer<float> o; NRVec<float> x=startpointf(4,1); bool conv; float f=runf(o,&tp,x,h,&conv); check("float COBYLA converges on testproblem n=4 (default rhoend)",conv && f<1e-5,f);}
+{Adam_hyperparameters<float> h; h.my_maxstep=20000; h.my_lr=1e-2; Adam_Optimizer<float> o; NRVec<float> x=startpointf(10,1); bool conv; float f=runf(o,&tp,x,h,&conv); check("float Adam converges on testproblem n=10",conv && f<1e-5,f);}
+{//finite-difference gradient with the float default step
+FiniteDifference<float> fd(&tp); NRVec<float> x=startpointf(10,2), g(10), gfd(10); tp.funcgrad(x,&g); fd.funcgrad(x,&gfd); gfd-=g;
+check("float FiniteDifference gradient (default step 5e-3)",gfd.norm()<1e-3*g.norm(),gfd.norm()/g.norm());
+FiniteDifference<double> fdd(&vd); FiniteDifference<float> fdf(&vf);
+compare_float<BFGS_Optimizer,BFGS_hyperparameters>("BFGS with finite-difference gradients (default steps)",&fdd,&fdf,8,3,5000);
+}
+{//Hessian-based methods on the VQE-like test problem, compared with double
+NRVec<double> xd=startpoint(6,3); NRVec<float> xf(6); for(int i=0;i<6;++i) xf[i]=xd[i];
+TrustRegion_hyperparameters<double> hd; TrustRegion_Optimizer<double> od; double fd=run(od,&vd,xd,hd);
+TrustRegion_hyperparameters<float> hf; TrustRegion_Optimizer<float> of; bool conv; float ff=runf(of,&vf,xf,hf,&conv);
+check("float trust region on testproblem_vqe agrees with double",conv && abs(ff-fd)<1e-5*max(1.,abs(fd)),ff-fd);
+}
+{//least squares: NIST Misra1a in float needs Marquardt scaling (unscaled b1 steps fall below float resolution)
+testproblem_misra1a<float> p; NRVec<float> b(2); for(int i=0;i<2;++i) b[i]=p.start(0,i);
+LM_hyperparameters<float> h; h.my_maxstep=100000; h.my_scaling=true; LM_Optimizer<float> o; bool conv; float f=runf(o,&p,b,h,&conv);
+float perr=0; for(int i=0;i<2;++i) perr=max(perr,(float)abs(b[i]-p.certified(i))/abs(p.certified(i)));
+check("float Levenberg-Marquardt fits NIST Misra1a",conv && perr<1e-3,perr);
+}
+}
+
 int main(int argc, char **argv)
 {
 test_problems_and_capabilities();
@@ -949,6 +1020,7 @@ test_leastsquares();
 test_maxfunc_and_parameters();
 test_diis();
 test_copyonwrite();
+test_float();
 cout <<"=== "<<npass<<" passed, "<<nfail<<" failed\n";
 return nfail;
 }

vec.cc

@@ -106,6 +106,23 @@ const NRVec<double> NRVec<double>::operator-() const {
 	return result;
 }
 
+template<>
+const NRVec<float> NRVec<float>::operator-() const {
+	NRVec<float> result(*this);
+	result.copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_sscal(nn, -1.0, result.v, 1);
+#ifdef CUDALA
+	}else{
+		cublasSscal(nn, -1.0, result.v, 1);
+		TEST_CUBLAS("cublasSscal");
+	}
+#endif
+	return result;
+}
+
 
 
 /***************************************************************************//**
@@ -129,6 +146,23 @@ const NRVec<std::complex<double> > NRVec<std::complex<double> >::operator-() con
 	return result;
 }
 
+template<>
+const NRVec<std::complex<float> > NRVec<std::complex<float> >::operator-() const {
+	NRVec<std::complex<float> > result(*this);
+	result.copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_csscal(nn, -1.0, result.v, 1);
+#ifdef CUDALA
+	}else{
+		cublasCsscal(nn, -1.0, (cuComplex*)result.v, 1);
+		TEST_CUBLAS("cublasCsscal");
+	}
+#endif
+	return result;
+}
+
 /***************************************************************************//**
  * unary minus operator for vector of general type
  * @return the modified vector
@@ -199,6 +233,16 @@ void NRVec<double>::randomize(const double &x){
 	}
 }
 
+template<>
+void NRVec<float>::randomize(const float &x){
+	NOT_GPU(*this);
+	copyonwrite();
+
+	for(register int i=0; i<nn; ++i){
+		v[i] = x*RANDDOUBLESIGNED();
+	}
+}
+
 /***************************************************************************//**
  * fill the complex vector with pseudorandom numbers generated using uniform distribution
  * the real and imaginary parts are generated independently
@@ -217,6 +261,16 @@ void NRVec<std::complex<double> >::randomize(const double &x) {
 	}
 }
 
+template<>
+void NRVec<std::complex<float> >::randomize(const float &x) {
+	NOT_GPU(*this);
+	copyonwrite();
+
+	for(register int i=0; i<nn; ++i){
+		v[i] = std::complex<float>(x*RANDDOUBLESIGNED(), x*RANDDOUBLESIGNED());
+	}
+}
+
 /***************************************************************************//**
  * constructor creating complex vector from a real one
  * @param[in] rhs the real vector being converted into the complex one
@@ -252,6 +306,31 @@ NRVec<std::complex<double> >::NRVec(const NRVec<double> &rhs, bool imagpart): nn
 #endif
 }
 
+template<>
+NRVec<std::complex<float> >::NRVec(const NRVec<float> &rhs, bool imagpart): nn(rhs.size()){
+
+	 count = new int;
+	*count = 1;
+#ifdef CUDALA
+	location = rhs.getlocation();
+	if(location == cpu){
+#endif
+		v = (std::complex<float>*)new std::complex<float>[nn];
+		memset(v, 0, nn*sizeof(std::complex<float>));
+		cblas_scopy(nn, &rhs[0], 1, ((float *)v) + (imagpart?1:0), 2);
+#ifdef CUDALA
+	}else{
+		v = (std::complex<float>*) gpualloc(nn*sizeof(std::complex<float>));
+
+		cublasCscal(nn, CUZERO, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasCscal");
+
+		cublasScopy(nn, &rhs[0], 1, ((float *)v) + (imagpart?1:0), 2);
+		TEST_CUBLAS("cublasScopy");
+	}
+#endif
+}
+
 /***************************************************************************//**
  * perform the <b>axpy</b> operation on the current real vector \f$\vec{v}\f$, i.e.
  * \f[ \vec{v} \leftarrow \vec{v} + \alpha\vec{x} \f]
@@ -278,6 +357,26 @@ void NRVec<double>::axpy(const double alpha, const NRVec<double> &x) {
 #endif
 }
 
+template<>
+void NRVec<float>::axpy(const float alpha, const NRVec<float> &x) {
+#ifdef DEBUG
+	if (nn != x.nn) laerror("incompatible vectors");
+#endif
+	SAME_LOC(*this, x);
+	copyonwrite();
+
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_saxpy(nn, alpha, x.v, 1, v, 1);
+#ifdef CUDALA
+	}else{
+		cublasSaxpy(nn, alpha, x.v, 1, v, 1);
+		TEST_CUBLAS("cublasSaxpy");
+	}
+#endif
+}
+
 /***************************************************************************//**
  * perform the <b>axpy</b> operation on the current complex vector \f$\vec{v}\f$, i.e.
  * \f[ \vec{v} \leftarrow \vec{v} + \alpha\vec{x} \f]
@@ -306,6 +405,28 @@ void NRVec<std::complex<double> >::axpy(const std::complex<double> alpha, const
 
 }
 
+template<>
+void NRVec<std::complex<float> >::axpy(const std::complex<float> alpha, const NRVec<std::complex<float> > &x){
+#ifdef DEBUG
+	if (nn != x.nn) laerror("incompatible vectors");
+#endif
+	SAME_LOC(*this, x);
+	copyonwrite();
+
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_caxpy(nn, &alpha, x.v, 1, v, 1);
+#ifdef CUDALA
+	}else{
+		const cuComplex _alpha = make_cuComplex(alpha.real(), alpha.imag());
+		cublasCaxpy(nn, _alpha, (cuComplex*)x.v, 1, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasCaxpy");
+	}
+#endif
+
+}
+
 /***************************************************************************//**
  * perform the <b>axpy</b> operation on the current real vector \f$\vec{v}\f$, i.e.
  * \f[ \vec{v} \leftarrow \vec{v} + \alpha\vec{x} \f]
@@ -321,6 +442,14 @@ void NRVec<double>::axpy(const double alpha, const double *x, const int stride){
 	cblas_daxpy(nn, alpha, x, stride, v, 1);
 }
 
+template<>
+void NRVec<float>::axpy(const float alpha, const float *x, const int stride){
+	NOT_GPU(*this);
+
+	copyonwrite();
+	cblas_saxpy(nn, alpha, x, stride, v, 1);
+}
+
 /***************************************************************************//**
  * perform the <b>axpy</b> operation on the current complex vector \f$\vec{v}\f$, i.e.
  * \f[ \vec{v} \leftarrow \vec{v} + \alpha\vec{x} \f]
@@ -336,6 +465,14 @@ void NRVec<std::complex<double> >::axpy(const std::complex<double> alpha, const
 	cblas_zaxpy(nn, &alpha, x, stride, v, 1);
 }
 
+template<>
+void NRVec<std::complex<float> >::axpy(const std::complex<float> alpha, const std::complex<float> *x, const int stride){
+	NOT_GPU(*this);
+
+	copyonwrite();
+	cblas_caxpy(nn, &alpha, x, stride, v, 1);
+}
+
 /***************************************************************************//**
  * assign real scalar value to every element of the current vector
  * @param[in] a scalar value to be assigned
@@ -357,6 +494,22 @@ copyonwrite();
 	return *this;
 }
 
+template<>
+NRVec<float>& NRVec<float>::operator=(const float &a){
+copyonwrite();
+
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_scopy(nn, &a, 0, v, 1);
+#ifdef CUDALA
+	}else{
+		smart_gpu_set(nn, (float)0, v);
+	}
+#endif	
+	return *this;
+}
+
 /***************************************************************************//**
  * assign complex scalar value to every element of the current vector
  * @param[in] a scalar value to be assigned
@@ -378,6 +531,22 @@ copyonwrite();
 	return *this;
 }
 
+template<>
+NRVec<std::complex<float> >& NRVec<std::complex<float> >::operator=(const std::complex<float> &a){
+copyonwrite();
+
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_ccopy(nn, &a, 0, v, 1);
+#ifdef CUDALA
+	}else{
+		smart_gpu_set(nn, (std::complex<float>)0, v);
+	}
+#endif	
+	return *this;
+}
+
 /***************************************************************************//**
  * assign scalar value to every element of the current vector of general type <code>T</code>
  * @param[in] a scalar value to be assigned
@@ -433,6 +602,38 @@ NRVec<double>& NRVec<double>::normalize(double *norm){
 	return *this;
 }
 
+template<>
+NRVec<float>& NRVec<float>::normalize(float *norm){
+	float tmp(0.0);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		tmp = cblas_snrm2(nn, v, 1);
+		if(norm) *norm = tmp;
+		#ifdef DEBUG
+			if(!tmp) laerror("attempt to normalize zero vector");
+		#endif
+		copyonwrite();
+		tmp = 1.0 / tmp;
+		cblas_sscal(nn, tmp, v, 1);
+#ifdef CUDALA
+	}else{
+		tmp = cublasSnrm2(nn, v, 1);
+		TEST_CUBLAS("cublasSnrm2");
+
+		if(norm) *norm = tmp;
+		#ifdef DEBUG
+			if(!tmp) laerror("attempt to normalize zero vector");
+		#endif
+		copyonwrite();
+		tmp = 1.0 / tmp;
+		cublasSscal(nn, tmp, v, 1);
+		TEST_CUBLAS("cublasSscal");
+	}
+#endif
+	return *this;
+}
+
 /***************************************************************************//**
  * normalize current complex vector (in the Euclidean norm)
  * @param[in] norm if not NULL, the norm of this vector is stored into *norm
@@ -471,6 +672,39 @@ NRVec<std::complex<double> > & NRVec<std::complex<double> >::normalize(double *n
 	return *this;
 }
 
+template<>
+NRVec<std::complex<float> > & NRVec<std::complex<float> >::normalize(float *norm){
+	float tmp(0.0);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		tmp = cblas_scnrm2(nn, v, 1);
+		if(norm) *norm = tmp;
+		#ifdef DEBUG
+		if(tmp == 0.0) laerror("attempt to normalize zero vector");
+		#endif
+		copyonwrite();
+		tmp = 1.0 / tmp;
+		cblas_csscal(nn, tmp, v, 1);
+#ifdef CUDALA
+	}else{
+		tmp = cublasScnrm2(nn, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasScnrm2");
+
+		if(norm) *norm = tmp;
+		#ifdef DEBUG
+			if(tmp == 0.0) laerror("attempt to normalize zero vector");
+		#endif
+		copyonwrite();
+
+		tmp = 1.0 / tmp;
+		cublasCsscal(nn, tmp, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasCsscal");
+	}
+#endif		
+	return *this;
+}
+
 /***************************************************************************//**
  * perform the \b gemv operation on this real vector \f$\vec{y}\f$, i.e.
  * \f[\vec{y}\leftarrow \alpha\operatorname{op}(A)\cdot\vec{x}+\beta\vec{y}\f]
@@ -503,6 +737,28 @@ void NRVec<double>::gemv(const double beta, const NRMat<double> &A,
 #endif
 }
 
+template<>
+void NRVec<float>::gemv(const float beta, const NRMat<float> &A, 
+		const char trans, const float alpha, const NRVec &x) {
+#ifdef DEBUG
+	if((tolower(trans) == 'n'?A.ncols():A.nrows()) != x.size()){ laerror("incompatible vectors"); }
+	if((tolower(trans) == 'n'?A.nrows():A.ncols()) != nn){ laerror("incompatible result vector in gemv"); }
+#endif
+	SAME_LOC3(*this, x, A);
+	copyonwrite();
+
+#ifdef CUDALA
+	if(location==cpu){
+#endif
+		cblas_sgemv(CblasRowMajor, (tolower(trans)=='n' ? CblasNoTrans:CblasTrans), A.nrows(), A.ncols(), alpha, A, A.ncols(), x.v, 1, beta, v, 1);
+#ifdef CUDALA
+	}else{
+		cublasSgemv((tolower(trans)=='n'?'T':'N'), A.ncols(), A.nrows(), alpha, A, A.ncols(), x.v, 1, beta, v, 1);
+		TEST_CUBLAS("cublasSgemv");
+	}
+#endif
+}
+
 /***************************************************************************//**
  * perform the \b gemv operation on this complex vector \f$\vec{y}\f$, i.e.
  * \f[\vec{y}\leftarrow \alpha\operatorname{op}(A)\cdot\vec{x}+\beta\vec{y}\f]
@@ -541,6 +797,34 @@ void NRVec<std::complex<double> >::gemv(const double beta, const NRMat<double> &
 #endif
 }
 
+template<>
+void NRVec<std::complex<float> >::gemv(const float beta, const NRMat<float> &A,
+                const char trans, const float alpha, const NRVec<std::complex<float> > &x) {
+#ifdef DEBUG
+	if ((tolower(trans) == 'n'?A.ncols():A.nrows()) != x.size()){ laerror("incompatible vectors"); }
+	if((tolower(trans) == 'n'?A.nrows():A.ncols()) != nn){ laerror("incompatible result vector in gemv"); }
+#endif
+	SAME_LOC3(*this, x, A);
+	copyonwrite();
+
+#ifdef CUDALA
+	if(location==cpu){
+#endif
+		cblas_sgemv(CblasRowMajor, (tolower(trans)=='n'?CblasNoTrans:CblasTrans),
+				A.nrows(), A.ncols(), alpha, A, A.ncols(), (float *)x.v, 2, beta, (float *)v, 2);
+		cblas_sgemv(CblasRowMajor, (tolower(trans)=='n'?CblasNoTrans:CblasTrans),
+				A.nrows(), A.ncols(), alpha, A, A.ncols(), ((float *)x.v) + 1, 2, beta, ((float *)v)+1, 2);
+#ifdef CUDALA
+	}else{
+		cublasSgemv((tolower(trans)=='n'?'T':'N'), A.ncols(), A.nrows(), alpha, A, A.ncols(), (float*)(x.v), 2, beta, (float *)v, 2);
+		TEST_CUBLAS("cublasSgemv");
+
+		cublasSgemv((tolower(trans)=='n'?'T':'N'), A.ncols(), A.nrows(), alpha, A, A.ncols(), ((float *)x.v) + 1, 2, beta, ((float *)v)+1, 2);
+		TEST_CUBLAS("cublasSgemv");
+	}
+#endif
+}
+
 
 /***************************************************************************//**
  * perform the \b gemv operation on this complex vector \f$\vec{y}\f$, i.e.
@@ -581,6 +865,35 @@ void NRVec<std::complex<double> >::gemv(const std::complex<double> beta,
 #endif
 }
 
+template<>
+void NRVec<std::complex<float> >::gemv(const std::complex<float> beta,
+		const NRMat<std::complex<float> > &A, const char trans, 
+		const std::complex<float> alpha, const NRVec<std::complex<float> > &x) {
+#ifdef DEBUG
+	if ((tolower(trans) == 'n'?A.ncols():A.nrows()) != x.size()){ laerror("incompatible vectors"); }
+	if((tolower(trans) == 'n'?A.nrows():A.ncols()) != nn){ laerror("incompatible result vector in gemv"); }
+#endif
+	SAME_LOC3(*this, x, A);
+	copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_cgemv(CblasRowMajor, (tolower(trans)=='n' ? CblasNoTrans : (tolower(trans)=='c'?CblasConjTrans:CblasTrans)), 
+				A.nrows(), A.ncols(), &alpha, A, A.ncols(), x.v, 1, &beta, v, 1);
+#ifdef CUDALA
+	}else{
+		if(tolower(trans)=='c') laerror("gemv with conjugate transposition not implemented on GPU");
+
+		const cuComplex _alpha = make_cuComplex(alpha.real(), alpha.imag());
+		const cuComplex  _beta = make_cuComplex(beta.real(), beta.imag());
+
+		cublasCgemv((tolower(trans)=='n'?'T':'N'), A.ncols(), A.nrows(),
+				_alpha, (cuComplex*)(A[0]), A.ncols(), (cuComplex*)(x.v), 1, _beta, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasCgemv");
+	}
+#endif
+}
+
 /***************************************************************************//**
  * perform the \b gemv operation on this real vector \f$\vec{y}\f$, i.e.
  * \f[\vec{y}\leftarrow \alpha\operatorname{op}(A)\cdot\vec{x}+\beta\vec{y}\f]
@@ -613,6 +926,28 @@ void NRVec<double>::gemv(const double beta, const NRSMat<double> &A,
 #endif
 }
 
+template<>
+void NRVec<float>::gemv(const float beta, const NRSMat<float> &A,
+		const char trans, const float alpha, const NRVec &x) {
+#ifdef DEBUG
+	if(A.ncols() != x.size()){ laerror("incompatible dimensions"); }
+	if(A.nrows() != nn){ laerror("incompatible result vector in gemv"); }
+#endif
+	SAME_LOC3(*this, A, x);
+	copyonwrite();
+
+#ifdef CUDALA
+	if(location==cpu){
+#endif
+		cblas_sspmv(CblasRowMajor, CblasLower, A.ncols(), alpha, A, x.v, 1, beta, v, 1);
+#ifdef CUDALA
+	}else{
+		cublasSspmv('U', A.ncols(), alpha, A, x.v, 1, beta, v, 1);
+		TEST_CUBLAS("cublasSspmv");
+	}
+#endif
+}
+
 
 /***************************************************************************//**
  * perform the \c gemv operation on this complex vector \f$\vec{y}\f$, i.e.
@@ -650,6 +985,32 @@ void NRVec<std::complex<double> >::gemv(const double beta, const NRSMat<double>
 #endif
 }
 
+template<>
+void NRVec<std::complex<float> >::gemv(const float beta, const NRSMat<float> &A,
+		const char trans, const float alpha, const NRVec<std::complex<float> > &x) {
+#ifdef DEBUG
+	if(A.ncols() != x.size()){ laerror("incompatible dimensions"); }
+	if(A.nrows() != nn){ laerror("incompatible result vector in gemv"); }
+#endif
+	SAME_LOC3(*this, A, x);
+	copyonwrite();
+
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_sspmv(CblasRowMajor, CblasLower, A.ncols(), alpha, A, (float *)x.v, 2, beta, (float *)v, 2);
+		cblas_sspmv(CblasRowMajor, CblasLower, A.ncols(), alpha, A, ((float *)x.v) + 1, 2, beta, ((float *)v) + 1, 2);
+#ifdef CUDALA
+	}else{
+		cublasSspmv('U', A.ncols(), alpha, A, (float*)(x.v), 2, beta, (float*)v, 2);
+		TEST_CUBLAS("cublasSspmv");
+		
+		cublasSspmv('U', A.ncols(), alpha, A, ((float*)(x.v)) + 1, 2, beta, ((float*)v) + 1, 2);
+		TEST_CUBLAS("cublasSspmv");
+	}
+#endif
+}
+
 /***************************************************************************//**
  * perform the \b gemv operation on this complex vector \f$\vec{y}\f$, i.e.
  * \f[\vec{y}\leftarrow \alpha\operatorname{op}(A)\cdot\vec{x}+\beta\vec{y}\f]
@@ -689,6 +1050,35 @@ void NRVec<std::complex<double> >::gemv(const std::complex<double> beta,
 #endif
 }
 
+template<>
+void NRVec<std::complex<float> >::gemv(const std::complex<float> beta,
+                const NRSMat<std::complex<float> > &A, const char trans,
+                const std::complex<float> alpha, const NRVec<std::complex<float> > &x){
+#ifdef DEBUG
+	if(A.ncols() != x.size()) laerror("incompatible dimensions");
+	if(A.nrows() != nn){ laerror("incompatible result vector in gemv"); }
+#endif
+	SAME_LOC3(*this, A, x);
+	copyonwrite();
+
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		if(tolower(trans) == 't') //row-major lower packed H is column-major upper packed H^T
+			cblas_chpmv(CblasColMajor, CblasUpper, A.ncols(), &alpha, A, x.v, 1, &beta, v, 1);
+		else
+			cblas_chpmv(CblasRowMajor, CblasLower, A.ncols(), &alpha, A, x.v, 1, &beta, v, 1);
+#ifdef CUDALA
+	}else{
+		const cuComplex _alpha = make_cuComplex(alpha.real(), alpha.imag());
+		const cuComplex  _beta = make_cuComplex(beta.real(), beta.imag());
+
+		cublasChpmv('U', A.ncols(), _alpha, (cuComplex*)((const std::complex<float>*)A), (cuComplex*)(x.v), 1, _beta, (cuComplex*)(this->v), 1);
+		TEST_CUBLAS("cublasChpmv");
+	}
+#endif
+}
+
 /***************************************************************************//**
  * computes the outer product of this real vector \f$\vec{a}\f$ with given
  * real vector \f$\vec{b}\f$ and scales the resulting matrix with factor \f$\alpha\f$, i.e.
@@ -716,6 +1106,24 @@ const NRMat<double> NRVec<double>::otimes(const NRVec<double> &b,const bool conj
 	return result;
 }
 
+template<>
+const NRMat<float> NRVec<float>::otimes(const NRVec<float> &b,const bool conj, const float &scale) const {
+
+	SAME_LOC(*this, b);
+	NRMat<float> result(0.0, nn, b.nn, this->getlocation());
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_sger(CblasRowMajor, nn, b.nn, scale, v, 1, b.v, 1, result, b.nn);
+#ifdef CUDALA
+	}else{
+		cublasSger(b.nn, nn, scale, b.v, 1, v, 1, result[0], b.nn);
+		TEST_CUBLAS("cublasSger");
+	}
+#endif
+	return result;
+}
+
 template<>
 const NRVec<double> NRVec<double>::otimes2vec(const NRVec<double> &b,const bool conj, const double &scale) const {
 
@@ -734,6 +1142,24 @@ const NRVec<double> NRVec<double>::otimes2vec(const NRVec<double> &b,const bool
 	return result;
 }
 
+template<>
+const NRVec<float> NRVec<float>::otimes2vec(const NRVec<float> &b,const bool conj, const float &scale) const {
+
+	SAME_LOC(*this, b);
+	NRVec<float> result(0.0, nn*b.nn, this->getlocation());
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_sger(CblasRowMajor, nn, b.nn, scale, v, 1, b.v, 1, result.v, b.nn);
+#ifdef CUDALA
+	}else{
+		cublasSger(b.nn, nn, scale, b.v, 1, v, 1, result.v, b.nn);
+		TEST_CUBLAS("cublasSger");
+	}
+#endif
+	return result;
+}
+
 /***************************************************************************//**
  * computes the outer product of this complex vector \f$\vec{a}\f$ with given
  * complex vector \f$\vec{b}\f$ and scales the resulting matrix with factor \f$\alpha\f$, i.e.
@@ -780,6 +1206,41 @@ NRVec<std::complex<double> >::otimes(const NRVec<std::complex<double> > &b, cons
 	return result;
 }
 
+template<>
+const NRMat<std::complex<float> > 
+NRVec<std::complex<float> >::otimes(const NRVec<std::complex<float> > &b, const bool conj, const std::complex<float> &scale) const {
+	
+	SAME_LOC(*this, b);
+	NRMat<std::complex<float> > result(0., nn, b.nn, this->getlocation());
+
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		if(conj){
+			cblas_cgerc(CblasRowMajor, nn, b.nn, &scale, v, 1, b.v, 1, result, b.nn);
+		}else{
+			cblas_cgeru(CblasRowMajor, nn, b.nn, &scale, v, 1, b.v, 1, result, b.nn);
+		}
+#ifdef CUDALA
+	}else{
+		if(conj){
+			const cuComplex alpha = make_cuComplex(scale.real(), -scale.imag());
+
+			cublasCgerc(b.nn, nn, alpha, (cuComplex*)(b.v), 1, (cuComplex*)(v), 1, (cuComplex*)(result[0]), b.nn);
+			TEST_CUBLAS("cublasCgerc");
+
+			result.conjugateme();
+		}else{
+			const cuComplex alpha = make_cuComplex(scale.real(), +scale.imag());
+
+			cublasCgeru(b.nn, nn, alpha, (cuComplex*)(b.v), 1, (cuComplex*)(v), 1, (cuComplex*)(result[0]), b.nn);
+			TEST_CUBLAS("cublasCgeru");
+		}
+	}
+#endif
+	return result;
+}
+
 template<>
 const NRVec<std::complex<double> > 
 NRVec<std::complex<double> >::otimes2vec(const NRVec<std::complex<double> > &b, const bool conj, const std::complex<double> &scale) const {
@@ -815,6 +1276,41 @@ NRVec<std::complex<double> >::otimes2vec(const NRVec<std::complex<double> > &b,
 	return result;
 }
 
+template<>
+const NRVec<std::complex<float> > 
+NRVec<std::complex<float> >::otimes2vec(const NRVec<std::complex<float> > &b, const bool conj, const std::complex<float> &scale) const {
+	
+	SAME_LOC(*this, b);
+	NRVec<std::complex<float> > result(0., nn*b.nn, this->getlocation());
+
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		if(conj){
+			cblas_cgerc(CblasRowMajor, nn, b.nn, &scale, v, 1, b.v, 1, result.v, b.nn);
+		}else{
+			cblas_cgeru(CblasRowMajor, nn, b.nn, &scale, v, 1, b.v, 1, result.v, b.nn);
+		}
+#ifdef CUDALA
+	}else{
+		if(conj){
+			const cuComplex alpha = make_cuComplex(scale.real(), -scale.imag());
+
+			cublasCgerc(b.nn, nn, alpha, (cuComplex*)(b.v), 1, (cuComplex*)(v), 1, (cuComplex*)(result.v), b.nn);
+			TEST_CUBLAS("cublasCgerc");
+
+			result.conjugateme();
+		}else{
+			const cuComplex alpha = make_cuComplex(scale.real(), +scale.imag());
+
+			cublasCgeru(b.nn, nn, alpha, (cuComplex*)(b.v), 1, (cuComplex*)(v), 1, (cuComplex*)(result.v), b.nn);
+			TEST_CUBLAS("cublasCgeru");
+		}
+	}
+#endif
+	return result;
+}
+
 
 template<>
 NRVec<std::complex<double> > complexify(const NRVec<double> &rhs) {
@@ -834,6 +1330,24 @@ NRVec<std::complex<double> > complexify(const NRVec<double> &rhs) {
 	return r;
 }
 
+template<>
+NRVec<std::complex<float> > complexify(const NRVec<float> &rhs) {
+	NRVec<std::complex<float> > r(rhs.size(), rhs.getlocation());
+
+#ifdef CUDALA
+	if(rhs.getlocation() == cpu){
+#endif
+		cblas_scopy(rhs.size(), &rhs[0], 1, (float *)(&r[0]), 2);
+#ifdef CUDALA
+	}else{
+		r = 0;
+		cublasScopy(rhs.size(), rhs.v, 1, (float*)(r.v), 2);
+		TEST_CUBLAS("cublasScopy");
+	}
+#endif
+	return r;
+}
+
 
 
 
@@ -1073,6 +1587,22 @@ const double NRVec<double>::sum() const {
         return result;
 }
 
+template <>
+const float NRVec<float>::sum() const {
+        float result=0;
+#ifdef CUDALA
+        if(location == cpu){
+#endif
+                        cblas_saxpy(nn, 1.0, v, 1, &result, 0);
+#ifdef CUDALA
+        }else{
+        laerror("not implemented");
+                }
+        }
+#endif
+        return result;
+}
+
 /***************************************************************************//**
  * sum up the all of the current double-precision complex vector
  * @return sum
@@ -1093,6 +1623,22 @@ const std::complex<double>  NRVec<std::complex<double> >::sum() const {
         return result;
 }
 
+template <>
+const std::complex<float>  NRVec<std::complex<float> >::sum() const {
+        std::complex<float>  result=0;
+#ifdef CUDALA
+        if(location == cpu){
+#endif
+                        cblas_caxpy(nn, &CONEF, v, 1, &result, 0);
+#ifdef CUDALA
+        }else{
+        laerror("not implemented");
+                }
+        }
+#endif
+        return result;
+}
+
 
 
 
@@ -1232,8 +1778,8 @@ INSTANTIZE_CONCAT(std::complex<double>)
 
 
 
-//template class NRVec<float>;
+template class NRVec<float>;
-//template class NRVec<std::complex<float> >;
+template class NRVec<std::complex<float> >;
 template class NRVec<double>;
 template class NRVec<std::complex<double> >;
 template class NRVec<char>;

vec.h

@@ -55,6 +55,7 @@ template <> inline unsigned long long MYABS(const  unsigned long long &x) {retur
  * static constants used in several cblas-routines
  ******************************************************************************/
 const static std::complex<double> CONE = 1.0, CMONE = -1.0, CZERO = 0.0;
+const static std::complex<float> CONEF = 1.0f, CMONEF = -1.0f, CZEROF = 0.0f;
 #ifdef CUDALA
 const static cuDoubleComplex CUONE = {1.,0.}, CUMONE = {-1.,0.}, CUZERO = {0.,0.};
 #endif
@@ -336,6 +337,7 @@ public:
 	void gemv(const T beta, const NRMat<T> &a, const char trans, const T alpha, const NRVec &x);
 	void gemv(const T beta, const NRSMat<T> &a, const char trans /**< just for compatibility reasons */, const T alpha, const NRVec &x);
 	void gemv(const T beta, const SparseMat<T> &a, const char trans, const T alpha, const NRVec &x,const bool treat_as_symmetric = false);
+	void gemv(const T beta, const CSRMat<T> &a, const char trans, const T alpha, const NRVec &x);
 
 	void gemv(	const typename LA_traits_complex<T>::Component_type beta, 
 			const typename LA_traits_complex<T>::NRMat_Noncomplex_type &a, 
@@ -375,6 +377,9 @@ public:
 		return result;
 	};
 
+	//! multiply given compressed row sparse matrix with this vector from left (defined in csrmat.h)
+	inline const NRVec operator*(const CSRMat<T> &mat) const;
+	
 	//! compute the outer product of two vectors 
 	const NRMat<T> otimes(const NRVec<T> &rhs, const bool conjugate = false, const T &scale = 1) const;
 
@@ -485,6 +490,7 @@ public:
 
 	//! constructor creating vector from sparse matrix
 	explicit NRVec(const SparseMat<T> &rhs);
+	explicit NRVec(const CSRMat<T> &rhs);
 	
 	//! routine for compatibility with sparse types
 	inline void simplify() {}; 
@@ -1517,6 +1523,7 @@ NRVec<std::complex<T> > complexify(const NRVec<T> &rhs) {
 	return r;
 }
 template<> NRVec<std::complex<double> > complexify<double>(const NRVec<double> &rhs);
+template<> NRVec<std::complex<float> > complexify<float>(const NRVec<float> &rhs);
 
 /***************************************************************************//**
  * routine for moving vector data between CPU and GPU memory
@@ -1580,6 +1587,24 @@ inline NRVec<double>& NRVec<double>::operator+=(const double &a) {
 	return *this;
 }
 
+template<>
+inline NRVec<float>& NRVec<float>::operator+=(const float &a) {
+	copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_saxpy(nn, 1.0, &a, 0, v, 1);
+#ifdef CUDALA
+	}else{
+		float *d = gpuputdouble(a);
+		cublasSaxpy(nn, 1.0, d, 0, v, 1);
+		TEST_CUBLAS("cublasSaxpy");
+		gpufree(d);
+	}
+#endif
+	return *this;
+}
+
 /***************************************************************************//**
  * adds a complex scalar value \f$\alpha\f$ to all elements of this complex vector \f$\vec{x}\f$
  * \f[\vec{x}_i\leftarrow\vec{x}_i+\alpha\f]
@@ -1604,6 +1629,24 @@ inline NRVec<std::complex<double> >& NRVec<std::complex<double> >::operator+=(co
 	return *this;
 }
 
+template<>
+inline NRVec<std::complex<float> >& NRVec<std::complex<float> >::operator+=(const std::complex<float> &a) {
+	copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_caxpy(nn, &CONEF, &a, 0, v, 1);
+#ifdef CUDALA
+	}else{
+		std::complex<float> *d = gpuputcomplex(a);
+                cublasCaxpy(nn, CUONE, (cuComplex *)d, 0, (cuComplex *)v, 1);
+		TEST_CUBLAS("cublasCaxpy");
+                gpufree(d);
+        }
+#endif
+	return *this;
+}
+
 /***************************************************************************//**
  * subtracts a real scalar value \f$\alpha\f$ from all elements of this real vector \f$\vec{x}\f$
  * \f[\vec{x}_i\leftarrow\vec{x}_i-\alpha\f]
@@ -1628,6 +1671,24 @@ inline NRVec<double>& NRVec<double>::operator-=(const double &a) {
 	return *this;
 }
 
+template<>
+inline NRVec<float>& NRVec<float>::operator-=(const float &a) {
+	copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_saxpy(nn, -1.0, &a, 0, v, 1);
+#ifdef CUDALA
+	}else{
+		float *d = gpuputdouble(a);
+		cublasSaxpy(nn, -1.0, d, 0, v, 1);
+		TEST_CUBLAS("cublasSaxpy");
+		gpufree(d);
+	}
+#endif
+	return *this;
+}
+
 /***************************************************************************//**
  * subtracts a complex scalar value \f$\alpha\f$ from all elements of this complex vector \f$\vec{x}\f$
  * \f[\vec{x}_i\leftarrow\vec{x}_i-\alpha\f]
@@ -1652,6 +1713,24 @@ inline NRVec<std::complex<double> >& NRVec<std::complex<double> >::operator-=(co
 	return *this;
 }
 
+template<>
+inline NRVec<std::complex<float> >& NRVec<std::complex<float> >::operator-=(const std::complex<float> &a) {
+	copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_caxpy(nn, &CMONEF, &a, 0, v, 1);
+#ifdef CUDALA
+	}else{
+		std::complex<float> *d = gpuputcomplex(a);
+		cublasCaxpy(nn, CUMONE, (cuComplex *)d, 0, (cuComplex *)v, 1);
+		TEST_CUBLAS("cublasCaxpy");
+		gpufree(d);
+	}
+#endif
+	return *this;
+}
+
 /***************************************************************************//**
  * adds a real vector \f$\vec{y}\f$ to this real vector \f$\vec{x}\f$
  * \f[\vec{x}\leftarrow\vec{x}+\vec{y}\f]
@@ -1678,6 +1757,26 @@ inline NRVec<double>& NRVec<double>::operator+=(const NRVec<double> &rhs) {
         return *this;
 }
 
+template<>
+inline NRVec<float>& NRVec<float>::operator+=(const NRVec<float> &rhs) {
+#ifdef DEBUG
+        if (nn != rhs.nn) laerror("incompatible dimensions");
+#endif
+	SAME_LOC(*this, rhs);
+        copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_saxpy(nn, 1.0, rhs.v, 1, v, 1);
+#ifdef CUDALA
+	}else{
+		cublasSaxpy(nn, 1.0, rhs.v, 1, v, 1);
+		TEST_CUBLAS("cubasDaxpy");
+	}
+#endif
+        return *this;
+}
+
 /***************************************************************************//**
  * adds a complex vector \f$\vec{y}\f$ to this complex vector \f$\vec{x}\f$
  * \f[\vec{x}\leftarrow\vec{x}+\vec{y}\f]
@@ -1704,6 +1803,26 @@ inline NRVec<std::complex<double> >& NRVec<std::complex<double> >::operator+=(co
 	return *this;
 }
 
+template<>
+inline NRVec<std::complex<float> >& NRVec<std::complex<float> >::operator+=(const NRVec<std::complex<float> > &rhs) {
+#ifdef DEBUG
+        if (nn != rhs.nn) laerror("incompatible dimensions");
+#endif
+	SAME_LOC(*this, rhs);
+        copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_caxpy(nn, &CONEF, rhs.v, 1, v, 1);
+#ifdef CUDALA
+	}else{
+		cublasCaxpy(nn, CUONE, (cuComplex*)rhs.v, 1, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasCaxpy");
+	}
+#endif
+	return *this;
+}
+
 /***************************************************************************//**
  * subtracts a real vector \f$\vec{y}\f$ from this real vector \f$\vec{x}\f$
  * \f[\vec{x}\leftarrow\vec{x}-\vec{y}\f]
@@ -1730,6 +1849,26 @@ inline NRVec<double> & NRVec<double>::operator-=(const NRVec<double> &rhs) {
         return *this;
 }
 
+template<>
+inline NRVec<float> & NRVec<float>::operator-=(const NRVec<float> &rhs) {
+#ifdef DEBUG
+        if (nn != rhs.nn) laerror("incompatible dimensions");
+#endif
+	SAME_LOC(*this,rhs);
+	copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_saxpy(nn, -1.0, rhs.v, 1, v, 1);
+#ifdef CUDALA
+	}else{
+		cublasSaxpy(nn, -1.0, rhs.v, 1, v, 1);
+		TEST_CUBLAS("cubasDaxpy");
+	}
+#endif
+        return *this;
+}
+
 /***************************************************************************//**
  * subtracts a complex vector \f$\vec{y}\f$ from this complex vector \f$\vec{x}\f$
  * \f[\vec{x}\leftarrow\vec{x}-\vec{y}\f]
@@ -1756,6 +1895,26 @@ inline NRVec<std::complex<double> >& NRVec<std::complex<double> >::operator-=(co
         return *this;
 }
 
+template<>
+inline NRVec<std::complex<float> >& NRVec<std::complex<float> >::operator-=(const NRVec<std::complex<float> > &rhs) {
+#ifdef DEBUG
+        if (nn != rhs.nn) laerror("incompatible dimensions");
+#endif
+	SAME_LOC(*this, rhs);
+	copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_caxpy(nn, &CMONEF, rhs.v, 1, v, 1);
+#ifdef CUDALA
+	}else{
+		cublasCaxpy(nn, CUMONE, (cuComplex*)rhs.v, 1, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasCaxpy");
+	}
+#endif
+        return *this;
+}
+
 /***************************************************************************//**
  * multiplies this real vector \f$\vec{x}\f$ by a real scalar value \f$\alpha\f$
  * \f[\vec{x}_i\leftarrow\alpha\vec{x}_i\f]
@@ -1778,9 +1937,28 @@ inline NRVec<double>& NRVec<double>::operator*=(const double &a) {
         return *this;
 }
 
+template<>
+inline NRVec<float>& NRVec<float>::operator*=(const float &a) {
+        copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_sscal(nn, a, v, 1);
+#ifdef CUDALA
+	}else{
+		cublasSscal(nn, a, v, 1);
+		TEST_CUBLAS("cublasSscal");
+	}
+#endif
+        return *this;
+}
+
 template<>
 inline NRVec<double>& NRVec<double>::operator/=(const double &a) {return *this *= (1./a);}
 
+template<>
+inline NRVec<float>& NRVec<float>::operator/=(const float &a) {return *this *= (1.f/a);}
+
 /***************************************************************************//**
  * multiplies this complex vector \f$\vec{x}\f$ by a complex scalar value \f$\alpha\f$
  * \f[\vec{x}_i\leftarrow\alpha\vec{x}_i\f]
@@ -1804,9 +1982,29 @@ inline NRVec<std::complex<double> >& NRVec<std::complex<double> >::operator*=(co
         return *this;
 }
 
+template<>
+inline NRVec<std::complex<float> >& NRVec<std::complex<float> >::operator*=(const std::complex<float> &a) {
+        copyonwrite();
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_cscal(nn, &a, v, 1);
+#ifdef CUDALA
+	}else{
+		const cuComplex alpha = make_cuComplex(a.real(), a.imag());
+		cublasCscal(nn, alpha, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasCscal");
+	}
+#endif
+        return *this;
+}
+
 template<>
 inline NRVec<std::complex<double> >& NRVec<std::complex<double> >::operator/=(const std::complex<double> &a) {return *this *= (1./a);}
 
+template<>
+inline NRVec<std::complex<float> >& NRVec<std::complex<float> >::operator/=(const std::complex<float> &a) {return *this *= (1.f/a);}
+
 /***************************************************************************//**
  * computes the inner product of this real vector \f$\vec{x}\f$ with given real vector \f$\vec{y]\f$
  * @param[in] rhs real vector \f$\vec{y}\f$
@@ -1832,6 +2030,26 @@ inline const double NRVec<double>::operator*(const NRVec<double> &rhs) const {
 	return ret;
 }
 
+template<>
+inline const float NRVec<float>::operator*(const NRVec<float> &rhs) const {
+	float ret(0.0);
+#ifdef DEBUG
+        if(nn != rhs.nn) laerror("incompatible dimensions");
+#endif
+	SAME_LOC(*this, rhs);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		ret = cblas_sdot(nn, v, 1, rhs.v, 1);
+#ifdef CUDALA
+	}else{
+		ret = cublasSdot(nn, v, 1, rhs.v, 1);
+		TEST_CUBLAS("cublasSdot");
+	}
+#endif
+	return ret;
+}
+
 /***************************************************************************//**
  * computes the inner product of this complex vector \f$\vec{x}\f$ with given complex vector \f$\vec{y}\f$
  * taking conjugation of vector \f$\vec{x}\f$ into account
@@ -1859,6 +2077,27 @@ inline const std::complex<double> NRVec<std::complex<double> >::operator*(const
         return dot;
 }
 
+template<>
+inline const std::complex<float> NRVec<std::complex<float> >::operator*(const NRVec< std::complex<float> > &rhs) const {
+#ifdef DEBUG
+        if(nn != rhs.nn) laerror("incompatible dimensions");
+#endif
+        std::complex<float> dot;
+	SAME_LOC(*this, rhs);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		cblas_cdotc_sub(nn, v, 1, rhs.v, 1, &dot);
+#ifdef CUDALA
+	}else{
+		const cuComplex val = cublasCdotc(nn, (cuComplex*)v, 1, (cuComplex*)rhs.v, 1);
+		TEST_CUBLAS("cublasCdotc");
+		dot = std::complex<float>(cuCreal(val), cuCimag(val)); 
+	}
+#endif
+        return dot;
+}
+
 /***************************************************************************//**
  * computes the inner product of this real vector \f$\vec{x}\f$ with given real data
  * @param[in] y pointer to the double-precision real array (sufficient length assumed)
@@ -1871,6 +2110,12 @@ inline const double NRVec<double>::dot(const double *y, const int stride, bool c
 	return cblas_ddot(nn,v, 1,y,stride);
 }
 
+template<>
+inline const float NRVec<float>::dot(const float *y, const int stride, bool conjugate) const {
+	NOT_GPU(*this);
+	return cblas_sdot(nn,v, 1,y,stride);
+}
+
 /***************************************************************************//**
  * computes the inner product of this complex vector \f$\vec{x}\f$ with given complex data
  * @param[in] y pointer to the double-precision complex array (sufficient length assumed)
@@ -1886,6 +2131,15 @@ inline const std::complex<double> NRVec<std::complex<double> >::dot(const std::c
         return dot;
 }
 
+template<>
+inline const std::complex<float> NRVec<std::complex<float> >::dot(const std::complex<float> *y, const int stride, bool conjugate) const {
+        std::complex<float> dot;
+	NOT_GPU(*this);
+        if(conjugate) cblas_cdotc_sub(nn, v,1,y, stride, &dot);
+	else cblas_cdotu_sub(nn, v,1,y, stride, &dot);
+        return dot;
+}
+
 /***************************************************************************//**
  * computes the sum of the absolute values of the elements of this real vector \f$\vec{x}\f$
  * @return \f$\sum_{i=1}^N\left|\vec{x}_i\right|\f$
@@ -1906,6 +2160,22 @@ inline const double NRVec<double>::asum() const {
 	return ret;
 }
 
+template<>
+inline const float NRVec<float>::asum() const {
+	float ret(0.0);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		ret = cblas_sasum(nn, v, 1);
+#ifdef CUDALA
+	}else{
+		ret = cublasSasum(nn, v, 1);
+		TEST_CUBLAS("cublasSasum");
+	}
+#endif
+	return ret;
+}
+
 
 /***************************************************************************//**
  * for this complex vector \f$\vec{x}\f$ compute the expression
@@ -1928,6 +2198,22 @@ inline const double NRVec<std::complex<double> >::asum() const {
 	return ret;
 }
 
+template<>
+inline const float NRVec<std::complex<float> >::asum() const {
+	float ret(0.0);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		ret = cblas_scasum(nn, v, 1);
+#ifdef CUDALA
+	}else{
+		ret = cublasScasum(nn, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasScasum");
+	}
+#endif
+	return ret;
+}
+
 /***************************************************************************//**
  * for this real vector \f$\vec{x}\f$ (of \f$N\f$ elements) determine the Frobenius norm
  * @return \f$\sum_{i=1}^N\left|\vec{x}_i\right|^2\f$
@@ -1948,6 +2234,22 @@ inline const double  NRVec<double>::norm(int length, int offset, int stride) con
 	return ret;
 }
 
+template<>
+inline const float  NRVec<float>::norm(int length, int offset, int stride) const {
+	float ret(0.);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		ret = cblas_snrm2((length>=0?length:nn), v+offset, stride);
+#ifdef CUDALA
+	}else{
+		ret = cublasSnrm2((length>=0?length:nn), v+offset, stride);
+		TEST_CUBLAS("cublasSnrm2");
+	}
+#endif
+	return ret;
+}
+
 /***************************************************************************//**
  * for this complex vector \f$\vec{x}\f$ (of \f$N\f$ elements) determine the Frobenius norm
  * @return \f$\sum_{i=1}^N\left|\vec{x}_i\right|^2\f$
@@ -1968,6 +2270,22 @@ inline const double NRVec< std::complex<double> >::norm(int length, int offset,
 	return ret;
 }
 
+template<>
+inline const float NRVec< std::complex<float> >::norm(int length, int offset, int stride) const {
+	float ret(0.);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		ret = cblas_scnrm2((length>=0?length:nn), v+offset, stride);
+#ifdef CUDALA
+	}else{
+		ret = cublasScnrm2((length>=0?length:nn), ((cuComplex*)v)+offset, stride);
+		TEST_CUBLAS("cublasSzrm2");
+	}
+#endif
+	return ret;
+}
+
 /***************************************************************************//**
  * for this real vector \f$\vec{x}\f$ determine the element with largest absolute value
  * @return \f$\vec{x}_i\f$ where \f$\left|\vec{x]_i\right|=\mathrm{max}_{j}\left|\vec{x}_{j}\right|\f$
@@ -1991,6 +2309,25 @@ inline const double NRVec<double>::amax() const {
 	return ret;
 }
 
+template<>
+inline const float NRVec<float>::amax() const {
+	if(nn==0) laerror("amax of an empty vector");
+	float ret(0.0);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		ret = v[cblas_isamax(nn, v, 1)];
+#ifdef CUDALA
+	}else{
+		const int pozice = cublasIsamax(nn, v, 1) ;
+		TEST_CUBLAS("cublasIsamax");
+
+		gpuget(1, sizeof(float), v + pozice, &ret);
+	}
+#endif
+	return ret;
+}
+
 template<>
 inline int NRVec<double>::iamax() const {
 	if(nn==0) laerror("iamax of an empty vector");
@@ -2008,6 +2345,23 @@ inline int NRVec<double>::iamax() const {
         return ret; 
 }       
 
+template<>
+inline int NRVec<float>::iamax() const {
+	if(nn==0) laerror("iamax of an empty vector");
+        int ret;
+#ifdef CUDALA
+        if(location == cpu){
+#endif  
+                ret = cblas_isamax(nn, v, 1);
+#ifdef CUDALA
+        }else{
+                ret = cublasIsamax(nn, v, 1) ;
+                TEST_CUBLAS("cublasIsamax");
+        }
+#endif  
+        return ret; 
+}       
+
 
 /***************************************************************************//**
  * for this real vector \f$\vec{x}\f$ determine the element with smallest absolute value
@@ -2039,6 +2393,32 @@ inline const double NRVec<double>::amin() const {
 	return ret;
 }
 
+template<>
+inline const float NRVec<float>::amin() const {
+	if(nn==0) laerror("amin of an empty vector");
+	float ret(0.0);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		//BLAS routine idamin seems no to be supported
+		float val(0.0);
+		int index(0);
+		ret = std::abs(v[0]);
+		for(register int i = 1; i < nn; i++){
+			val = std::abs(v[i]);
+			if(val < ret){ index = i; ret = val; }
+		}
+		ret = v[index];
+#ifdef CUDALA
+	}else{
+		const int pozice = cublasIsamin(nn, v, 1);
+		TEST_CUBLAS("cublasIsamin");
+		gpuget(1, sizeof(float), v + pozice, &ret);
+	}
+#endif
+	return ret;
+}
+
 /***************************************************************************//**
  * for a given complex vector \f$\vec{v}\f$, determine the smallest index of the maximum 
  * magnitude element, i.e. maximal element in the 1-norm
@@ -2062,6 +2442,24 @@ inline const std::complex<double> NRVec<std::complex<double> >::amax() const {
 	return ret;
 }
 
+template<>
+inline const std::complex<float> NRVec<std::complex<float> >::amax() const {
+	if(nn==0) laerror("amax of an empty vector");
+	std::complex<float> ret(0., 0.);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		ret = v[cblas_icamax(nn, v, 1) ];
+#ifdef CUDALA
+	}else{
+		const int pozice = cublasIcamax(nn, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasIcamax");
+		gpuget(1, sizeof(std::complex<float>), v + pozice, &ret);
+	}
+#endif
+	return ret;
+}
+
 template<>
 inline int NRVec<std::complex<double> >::iamax() const {
 	if(nn==0) laerror("iamax of an empty vector");
@@ -2079,6 +2477,23 @@ inline int NRVec<std::complex<double> >::iamax() const {
         return ret;
 }
 
+template<>
+inline int NRVec<std::complex<float> >::iamax() const {
+	if(nn==0) laerror("iamax of an empty vector");
+	int ret;
+#ifdef CUDALA
+        if(location == cpu){
+#endif
+                ret = cblas_icamax(nn, v, 1);
+#ifdef CUDALA
+        }else{
+                ret = cublasIcamax(nn, (cuComplex*)v, 1);
+                TEST_CUBLAS("cublasIcamax");
+        }
+#endif
+        return ret;
+}
+
 
 /***************************************************************************//**
  * for a given complex vector \f$\vec{v}\f$, determine the smallest index of the minimum 
@@ -2113,6 +2528,34 @@ inline const std::complex<double> NRVec<std::complex<double> >::amin() const {
 	return ret;
 }
 
+template<>
+inline const std::complex<float> NRVec<std::complex<float> >::amin() const {
+	if(nn==0) laerror("amin of an empty vector");
+	std::complex<float> ret(0., 0.);
+#ifdef CUDALA
+	if(location == cpu){
+#endif
+		// izamin seems not to be supported
+		int index(0);
+		float val(0.0), min_val(std::numeric_limits<float>::max());
+		std::complex<float> z_val(0.0, 0.0);
+
+		for(register int i=0; i < nn; i++){
+			z_val = v[i];
+			val = std::abs(z_val.real()) + std::abs(z_val.imag());
+			if(val < min_val){ index = i; min_val = val; }	
+		}
+		ret = v[index];
+#ifdef CUDALA
+	}else{
+		const int pozice = cublasIcamin(nn, (cuComplex*)v, 1);
+		TEST_CUBLAS("cublasIcamin");
+		gpuget(1, sizeof(std::complex<float>), v + pozice, &ret);
+	}
+#endif
+	return ret;
+}
+
 
 /***************************************************************************//**
  * routine for raw output