professor

view ChangeLog @ 556:3ded80f33ed3

calculate unbiased chi2 after minimsation finishes and write to results file
author Holger Schulz <holger.schulz@durham.ac.uk>
date Thu, 05 Apr 2018 13:44:54 -0500
parents 1e8d27ae9d7c
children
line source
1 2018-02-07 Andy Buckley <andy.buckley@cern.ch>
3 * Add numCoeffs, numParams, and exprString methods to Ipol.
5 2017-10-25 Holger Schulz <iamholger@gmail.com>
7 * Bugfix missing import statement
9 2017-12-29 Holger Schulz <iamholger@gmail.com>
11 * Exposed more C++ functions to python for Pade approximation testing
13 2017-11-10 Holger Schulz <iamholger@gmail.com>
15 * Release 2.2.2 --- this is the result of many months of fiddling
16 with especially multinest requirements.
18 * Sampling --- sobol, latin hypercube and random uniform sampler
19 --- something like PARAM 0 1 2 3 result in sub space
20 sampling
21 --- biases now work e.g. PARAM 0 1 exp(x)
22 * Ipol --- --order auto and friends automatically determine
23 best order, this avoids oversampling.
24 --- --logy parmeterises the logy of things, useful
25 for large paramspace and especially likelihood
26 --- --medianfilt can drop anchor points based on error
27 this is experimental and meant to deal with the
28 occasional huge NLO weight
29 --- The output has the valid region for each object now
30 stored at the end of the coefficients, this is also where
31 they are read from --- this allows the usage of
32 --medianfilt etc
33 * Tune --- Replace the grid sampling with the already existing
34 sampler machinery, so can use random, latin, sobol when
35 using --scan to find good start point for minuit
36 --- --profiles and --contours stores the minuit and minos
37 plots
38 --- limits are by default now set to the ipol region, this
39 can be overridden when using -x CL arg
40 --- tune script can read more than one ipol file
41 A lot of stuff has been put into libraries to allow reusing
42 individual steps.
44 * Multinest --- this is now supported in prof2-tune-nest
45 * GP --- gp_minimize via prof2-tune-gp
46 * emcee --- MCMC sampling via prof2-emcee
48 * Eigentunes --- prof2-eigentunes is back and working, deltaChi2
49 must be provided, however, via --target
51 * Bootstrapping in principle working but smearing is probably
52 more complicated than what we have now prof2-bootstrap
56 2017-08-22 Holger Schulz <holger.schulz@durham.ac.uk>
58 * Improvements to Ipol --- we allow different orders for each bin now,
59 there is also --order auto which does some jackknifing to figure out
60 the best suited polynomial order to avoid overfitting. The Ipol
61 outputformat is now binned 3 which dumps the minparam values and
62 maxparamvalues at the end of the coefficient string. This is necessary
63 as we now also allow to filter the inputs by median error (necessary
64 to tame NLO spiky histograms). The ranges are read back for each bin
65 separately now, happens in the C++ part.
67 * Overhaul of sampling, including things such as full support of sobol
68 sampling. Also, param range files with more than two numbers on a line
69 are interpreted as sub spaces for patch sampling.
71 * Add script prof2-ipolenvelopes which for each bin finds the minimum
72 and maximum polynomial value (using migrad) in the range of the
73 ipols validity. This is nice especially to check envelopes once
74 filtering in the ipol step was done
76 2017-05-18 Holger Schulz <holger.schulz@durham.ac.uk>
78 * Refitting of ipols using smeared ipol output.
79 Required some modification in data io structure.
81 2017-05-04 Holger Schulz <holger.schulz@durham.ac.uk>
83 * Fix in prof2-tune when checking analysis names, at the same time
84 allow for for closure test tuning, i.e. tune against input folder.
87 2017-05-02 Holger Schulz <holger.schulz@durham.ac.uk>
89 * Use -march=native according to eigen3 faq for better performance
90 as well as -O3 in pyext
92 * Replace modulus based status message by simple counter in ipol stage
94 * Set version to 2.2.2beta for imminent release
96 2017-04-28 Holger Schulz <holger.schulz@durham.ac.uk>
98 * Use BDSSVD in case Eigen >=3.3.3
100 * Reworked prof2-ipol and ipol.py to use mk_ipolbin rather than
101 mk_ipolhisto, some speedups plus additional benefit of being
102 able to fiddle with inividual bins in the ipol building stage
104 * add --minos switch to prof2-tune toi run MINOS
106 * Write GOF and NDOF into result file
108 2017-02-21 Andy Buckley <andy.buckley@cern.ch>
110 * Change prof2-ipol ierr mode default to 'symm'
112 2017-02-17 Andy Buckley <andy.buckley@cern.ch>
114 * Add a --limit-errs argument for prof2-tune, to avoid annoying
115 re-reading of large run dirs
117 * Tweaks to default error ipol order, and release figure handles
118 in envelope plotting.
120 2017-02-02 Holger Schulz <holger.schulz@durham.ac.uk>
122 * Update Copyright string
124 * Instead of exiting when prof2-ipol encounters nan coefficients in
125 value or error ipols, spit out a warning and ignore that histogram
126 entirely --- this makes prof2-ipol and friends much more robust
128 * tag as 2.2.1 and release.
130 2017-02-01 Holger Schulz <holger.schulz@durham.ac.uk>
132 * Use Agg as default matplotlib backend.
134 * Bump version to 2.2.1
136 2016-12-19 Holger Schulz <holger.schulz@cern.ch>
138 * Even more robust prof-tune, added --filter CL option to
139 automatically get rid of zero error bins
141 * Fixed a bug where IpolBin.n was never set which is needed for
142 the PointMatcher (i.e. weights syntax)
144 * Not using YODA Pointmatcher but own due to different regexes
145 required.
147 * Make release 2.2
149 2016-12-15 Holger Schulz <holger.schulz@cern.ch>
151 * More robust scripts and more meaningful error messages, especially
152 prof2-tune
154 2016-11-30 Holger Schulz <holger.schulz@cern.ch>
156 * Replace Counter code when running mk_Structure with much faster code
158 * Bump version to 2.2
160 * Bump DataFormat to binned 2
162 * Allow passing file names or open files (or similar streams) to
163 ipolio functions
165 * prof2-ipol can read yaml now
167 * Add some of Elliot's code for choosing sub spaces in prof2-ipol and
168 some plot routines to contrib
170 * Add SOBOL sampling to contrib
172 2016-08-26 Holger Schulz <holger.schulz@cern.ch>
174 * Use more readable Cython DL path in bootstrap
176 2016-07-18 Andy Buckley <andy.buckley@cern.ch>
178 * Typo fixes -- thanks to Leif Gellersen.
180 2016-07-14 Andy Buckley <andy.buckley@cern.ch>
182 * Release version 2.1.4
184 * Improve Makefile tests for ROOT and Cython versions.
186 2016-07-11 Andy Buckley <andy.buckley@cern.ch>
188 * Add Ipol min/max param vals getters and map them into Python.
190 * Add a built-in /REF-stripping option to data histo loading.
192 * Improvements to the Python ipol metadata interface.
194 2016-07-05 Andy Buckley <andy.buckley@cern.ch>
196 * Try to optimise long-vector building by not executing pow() if the exponent would be zero.
198 * Split C++ calcValue function into two, to allow re-use of a precalculated long-vector.
200 2016-06-23 Andy Buckley <andy.buckley@cern.ch>
202 * Split 'dot product' part of ipol value calculation into a standalone calcValue function.
204 * Use const references when passing const vectors in C++ API.
206 2016-04-14 Holger Schulz <holger.schulz@cern.ch>
208 * Introduce 1D sensitivity based on gradient
210 * User needs to specify --cmap or --grad when running prof2-sens now
212 2016-04-01 Holger Schulz <holger.schulz@cern.ch>
214 * Add a first version of the parametrisation using Machine Learning
216 2016-02-16 Holger Schulz <holger.schulz@cern.ch>
218 * Bump to version 2.1.3
220 2016-02-15 Holger Schulz <holger.schulz@cern.ch>
222 * prof2-sens: bugfix, add weight file usage, clean up CL
224 * prof2-I: add weight file usage
226 * prof2-predict: add weight file usage, fix output writing
228 * Extended documentation
230 2016-02-12 Holger Schulz <holger.schulz@cern.ch>
232 * Documentation including asciinema
234 2016-02-11 Holger Schulz <holger.schulz@cern.ch>
236 * Remove min_runs function, replace with numCoeffs
237 from C++ code
239 * Add prof2-envelopes script
241 * In prof-tune simplechi2 --- continue on weight==0
242 to allow some robustness when switching off bins
245 2016-02-10 Holger Schulz <holger.schulz@cern.ch>
247 * First version of prof2-runcombs plus sampling functions
248 salvaged from Prof1, modified to have no scipy dependence
250 * Add --rc switch to prof2-ipol to optionally read runcombs
251 Syntax is --rc runcoms.dat[colon]5
252 (Minimally invasive addon)
254 * Code cull in prof2-ipol. Only support multiprocessing
255 version, default 1 thread. Changed CL switch to '-j'.
258 2015-12-16 Holger Schulz <holger.schulz@cern.ch>
260 * Only calculate polynomials structure once per instance
261 -> significant speed increase
263 * Add the brute force grid scan (--scan) to prof2-tune
266 2015-12-11 Holger Schulz <holger.schulz@cern.ch>
268 * Minimiser tweaks --- no more need for param translation,
269 make strategy steerable (--strategy or -s), -q suppresses
270 iminuit output now, calculation and writing of correlation
271 matrix into results, dump weights into results file
273 2015-12-10 Holger Schulz <holger.schulz@cern.ch>
275 * Undo ui madness
277 * Remove numpy dependence (except sampling.py)
279 * Add prof-sens for sensitivity plotting exploting derivatives
281 2015-12-09 Holger Schulz <holger.schulz@cern.ch>
283 * Can specify PREFIX when calling make
285 2015-12-09 Andy Buckley <andy.buckley@cern.ch>
287 * Improve prof2-predict UI.
289 * Add metadata printing in prof2-lsipol debug mode.
291 * Add -v and -q flag shortcuts to all scripts.
293 2015-12-07 Holger Schulz <holger.schulz@cern.ch>
295 * Use __all__ in __init__.py
297 * Started sphinx documentation
299 * Support only iminuit, bump version to 2.2.alpha, restructure imports
301 * Sanity checks when calling parametrisation
303 2015-12-03 Andy Buckley <andy.buckley@cern.ch>
305 * Provide a version() function in the C++ library, and map it into
306 Python. Both sources use the VERSION variable defined in the
307 Makefile at build time.
309 * Reduce SVD fitting threshold to 1e-20.
311 2015-12-03 Holger Schulz <holger.schulz@cern.ch>
313 * Make derivative calculation work also when scaling params
315 * Let YODA write YODA
317 2015-12-02 Holger Schulz <holger.schulz@cern.ch>
319 * Derivative calculation
321 2015-11-28 Andy Buckley <andy.buckley@cern.ch>
323 * Version 2.1.2
325 * Revert range shift in Ipol.cc, since it produced (small)
326 numerical errors rather than improved stability. To be
327 understood...
329 * Add test/mkpolydata script, for generating polynomial pseudodata
330 used in closure tests.
332 2015-11-27 Andy Buckley <andy.buckley@cern.ch>
334 * Fix bug in histo loading.
336 * Adding log binning and other options to prof2-residuals.
338 2015-11-24 Andy Buckley <andy.buckley@cern.ch>
340 * Version 2.1.1 release.
342 * Fix prof2-predict YODA writing, and improve DataHisto.toYODA.
344 * Change parameter rescaling from using the [0,1] range to instead
345 use [1,2], to avoid arbitrarily scaled param numbers. BREAKS IPOL
346 FILE COMPATIBILITY WITH 2.1.0!!
348 * Set minimizer initial step size = prange/10.
350 * Add automatic determination of maximum MC errors in prof-tune,
351 and use them to regularize interpolated errs which could distort
352 the fit.
354 * Separate internal Minuit function names (with syntax
355 restrictions) from the free-form external param names.
357 2015-11-23 Andy Buckley <andy.buckley@cern.ch>
359 * Add pyext/professor2/chi2.py
361 * Add find_maxerrs() to the Python interface, for use in error regularisation.
363 * Use the param file name as a regex.
365 * Use __slots__ in the internal histogram and bin types, for better memory efficiency.
367 * Fix (re-fix?) use of inappropriate indices in calculation of mean and median MC uncertainties.
369 * More proactive data object deletion after conversion to YODA data format.
371 2015-11-22 Andy Buckley <andy.buckley@cern.ch>
373 * Version 2.1.0 (significant version change due to scaling, all-orders, and ROOT support)
375 2015-11-18 Andy Buckley <andy.buckley@cern.ch>
377 * dataio.py: Add ROOT file reading ability.
379 * Change rescaling I/O behaviour a bit, moving some hacked
380 Python-only Ipol string constructors into explicit API improvements, and
381 only activating rescaling if DoParamScaling is set true.
383 2015-11-16 Andy Buckley <andy.buckley@cern.ch>
385 * Add CPPFLAGS steering in Makefile and API tidying.
387 2015-11-11 Holger Schulz <holger.schulz@cern.ch>
389 * Add automatic raw param value -> unit range mapping to Ipol.
391 * Add any-order long vector calculation.
393 2015-11-10 Andy Buckley <andy.buckley@cern.ch>
395 * Add prof2-lsipol as a handy way to make a tuning weights file
396 and for general convenience.
398 2015-10-06 Andy Buckley <andy.buckley@cern.ch>
400 * Version 2.0.0
402 * Add 'make dist' target to Makefile, and tweak/reduce Cython necessity
404 * Allow calls to IpolBin and IpolHisto methods with params passed
405 as unpacked *args as well as a packed iterable.
407 * Add possibility to pass params as an dict-like as well as a list
408 or tuple of floats -- note that the dict-like must return
409 *ordered* values consistent with the Ipol training. Also ensure
410 the float type.
412 * Add conversion of C++ exceptions to Python exceptions.
414 * Add professor2.utils module, providing an opportunistic use of
415 OrderedDict for params so the params order required by Ipol is the
416 same as the order in which they are specified in the (first run's)
417 params.dat file.
419 * Change Python-mapped Ipol methods with no args to be properties.
421 * Add nice __repr__ methods for Histo and Bin objects.
423 * Specialise Histo as IpolHisto and DataHisto, and provide
424 toData{Histo,Bin} on IpolHisto and IpolBin respectively.
426 * Remove ProfMaster.
428 2015-10-04 Andy Buckley <andy.buckley@cern.ch>
430 * Add protection of svd.setThreshold call to ensure it is
431 supported/needed, via Eigen version number macros.
433 * Add vmin and vmax args to Ipol and IpolBin in Python.
435 2015-10-02 Andy Buckley <andy.buckley@cern.ch>
437 * Put histo file reading in a try..except block so read failures
438 on non-histo files in the run dirs are not fatal.
440 * Add protection in the histo loader, so we don't trip up on
441 Rivet's new cross-section and counter objects.
443 2015-09-30 Andy Buckley <andy.buckley@cern.ch>
445 * src/Ipol.h: Use Eigen/SVD include path rather than non-standard eigen3/Eigen/SVD.
447 2015-09-14 Holger Schulz <holger.schulz@durham.ac.uk>
449 * Use Eigen's svd.setThreshold(1e-20) to prevent the errors we saw with Eigen
450 3.2.2 and later when looking at 5th order polynomials in 3 dimensinos
451 with 900+ anchors
453 * Some clean up
455 2015-09-11 Andy Buckley <andy.buckley@cern.ch>
457 * Adding prof2-residuals script for ipol faithfulness testing.
459 * Rename scripts from prof-* to prof2-* to allow parallel installations of Prof1 and Prof2.
461 * Move some non-core scripts from bin to contrib.
463 * Make root-config dependency optional
465 * Lots of reworking to make prof-tune work again, and to use weight file parsing.
467 2015-09-10 Andy Buckley <andy.buckley@cern.ch>
469 * Use weight file path parsing in prof-ipol.
471 * Add prof.weights submodule, with a copy of PointMatcher and a
472 new read_pointmatchers function for handling weight files.
474 * Provide __version__attribute in the Python module.
476 * User script CLI consistency, simplification, general smoothing...
478 2015-08-30 Andy Buckley <andy.buckley@cern.ch>
480 * Add recursive scangrid generator function, and other tweaks.
482 2015-08-18 Andy Buckley <andy.buckley@cern.ch>
484 * Tweaks to ipol file parsing functions.
486 * Fix a bug in IpolBin's use of interpolated errors.
488 * Convert mean and median error parameterisation to use new 0th order ipols.
490 2015-08-15 Holger Schulz <holger.schulz@durham.ac.uk>
492 * A first version of catching singular matrices in the SVD
494 * Ipol has 1 additional argument, "threshold", that determines what
495 singular values are considered 0, fully propagated to pyext
497 * Going 2 Pro 2 Fessional
499 * Some startup checks of prof- scripts
501 * remove bin/prof-sampling, rename prof-prediction -> prof-predict
503 2015-08-13 Holger Schulz <holger.schulz@durham.ac.uk>
505 * Reverted unintended commit of hack.
507 * Added very basic prof-config script to bin
509 * Update PATH when sourcing setup.sh
511 2015-08-12 Holger Schulz <holger.schulz@durham.ac.uk>
513 * Added prof-prediction which reads in a ifile and a point in
514 parameter space (either a text file, a comma separated list or just
515 the args) and writes out an ipolhisto. Tested to work.
517 2015-08-03 Holger Schulz <holger.schulz@durham.ac.uk>
519 * Added 0-order polynomials, i.e. constant values.
520 Currently, the coefficent is simply the value of
521 the first anochorpoint.
523 2015-07-15 Holger Schulz <holger.schulz@durham.ac.uk>
525 * Some pull plotting functionality
527 2015-07-08 Holger Schulz <holger.schulz@durham.ac.uk>
529 * Calculate and print G.o.f.
531 * Write some meta info to results
533 * Read limits, fixed params from single textfile --limits
535 * Output files steered by --output
537 * Remove now obsolete bin/prof-interpolate
540 2015-07-08 Holger Schulz <holger.schulz@durham.ac.uk>
542 * First working version of prof-tune (pyminuit, simpleGOF)
544 * Multiprocessing to speed up prof-ipol (--multi)
546 * prof-ipol-tabulated to write out interpolation of tabulated data
548 * Minor bugfixes
551 2015-06-16 Andy Buckley <andy.buckley@cern.ch>
553 * Adding mean and median strategies for assigning constant ipol bin errors.
555 2015-06-03 Andy Buckley <andy.buckley@cern.ch>
557 * Adding a Bin base class in the Python side, and a coherent handling of multiple (interpolated) bin errors.
559 * Various tweaks and minor fixes.
561 2015-05-10 Andy Buckley <andy.buckley@cern.ch>
563 * Add 'Minuit' class importing to the __init__.py file.
565 2015-05-05 Andy Buckley <andy.buckley@cern.ch>
567 * Rewrite C++ ipolstring parsing using STL string functions rather than Boost: shorter! Boost dependency GONE :-)
569 * Replace boost::tuple with std::pair and clean up ParamPoints interface a bit.
571 * Add a --veto option to prof-sample, for a user-specified vetoing function.
573 * Move Sampler from prof-sample into professor2/__init__.py
575 2015-05-04 Andy Buckley <andy.buckley@cern.ch>
577 * Adding a first version of prof-tune with ipol reading from file.
579 * Adding options for (average) error interpolation and ipol persistency to prof-ipol and module functions.
581 2015-05-03 Andy Buckley <andy.buckley@cern.ch>
583 * Add bin/prof-ipol simple script using the functions below -- to be merged with prof-interpolate.
585 * professor2/__init__.py: Add Histo, DataBin, IpolBin and basic data handling functions.
587 * Remove unnecessary dlopen fiddling from professor2/__init__.py
589 2015-04-23 Andy Buckley <andy.buckley@cern.ch>
591 * Remove bound ParamPoints pointer from Ipol, and ditch lazy coeff evaluation in favour of simplicity and construct-time coeff calculating.
593 * Move long vector and coeff building into unbound functions rather than Ipol methods.
595 * Start of ChangeLog. Library has already iterated quite a bit.
597 * FILL IN EARLY HISTORY FROM HG LOG