common-lisp-stat
文件大小: unknow
源码售价: 5 个金币 积分规则     积分充值
资源说明:Common Lisp Statistics -- based on LispStat (Tierney) but updated for Common Lisp and incorporating lessons from R (http://www.r-project.org/). See the google group for lisp stat / common lisp statistics for a mailing list.
Time-stamp: <2014-04-07 16:28:57 tony>

* Current Status: IMPROVING

  but we are rebuilding it.

* Fast Start

  Here's a general fast start approach for using this.   

  1. Get access to a Common Lisp implementation on your platform.
     Make sure you have BLAS, LAPACK, and their corresponding
     development environments on your system.

     For example, on a recent debian (testing), one may need:
#+BEGIN_EXAMPLE
libffi6
libblas
liblapack
libgsl
#+END_EXAMPLE
     and the corresponding /-dev/ packages.

  2. if needed, install quicklisp (http://www.quicklisp.org)
  3. if wanted, install git (http://www.git-scm.org/)
  4. Use git to fetch common-lisp-statistics from the repository:
        git://github.com/blindglobe/common-lisp-stat.git
     and put it into your quicklisp local-projects directory
     (you will need to put a few more projects there as well), 
     OR make sure you have an internet connection and go to step 5 and
     fetch a not-so-bleeding-edge version from QuickLisp.

     We suggest something like:

#+begin_src shell
mkdir ~/quicklisp/
mkdir ~/quicklisp/local-projects
cd ~/quicklisp/local-projects
git clone git://github.com/blindglobe/common-lisp-stat.git
git clone git://github.com/blindglobe/lisp-matrix.git
#+end_src

     (with the 5th and on lines representing stuff undergoing rapid
     change, so possibly out of date on QUICKLISP).

     You might need to grab XARRAY as well, and if behind a firewall,
     might need to use HTTP or similar alternative transport.

  5. Start up the Common Lisp implementation, and:

#+name: loadIt
#+begin_src lisp
  (ql:register-local-projects)
  (ql:quickload :antik)
  (ql:quickload :cls)
#+end_src

     TODO: Unfortunately, it looks like ANTIK needs to be preloaded,
     or it gets confused.  I don't yet know why.

  6. get coffee, tea, water, beer, or a glass of wine, review the
     Common Lisp Statistics mailing list, chat with friends, etc,
     until it is done compiling and loading.
  7. Report success/failure back to the news group (
      https://groups.google.com/forum/?fromgroups#!forum/lisp-stat )
     or directly to Tony Rossini mail, i.e.  (
     mailto:blindglobe@gmail.com ) if you don't want to be public
     about it.
  8. Start trying it out.  The examples/  directory is a good place
     to start evaluating the files, one at a time.  Preferrably in
     order! 

  More details can be fond in the subsequent sections.

** Common Lisp

   Current requirements are basically that you have a Common Lisp
   implementation such as SBCL or CCL (though CLISP might work) and
   BLAS/LAPACK on your system.

   SBCL and CCL are in active use by developers.  We are interested in
   finding out details regarding successes with other distributions
   and would be pleased to add such information here.


*** TODO Microsoft Windows

    Currently, LispCabinet would be the simplest way to get started,
    with either SBCL or CCL as the Lisp Implementation.  Need
    experiences from others using 

    http://lispcabinet.sourceforge.net/

*** Linux

    Most distributions contain SBCL or CLISP.  The instructions for
    getting and installing CCL are relatively straightforward.

*** MacOSX

    Both CCL and SBCL have been remarked to work.  CCL seems to be
    preferred on this platform?

** Configuring BLAS/LAPACK libraries locations

   Currently this is a manual operation, which will change in a later
   version.

   Edit the file external/cl-blapack/load-blapack-libs.lisp

   Search for the following 3 parameters 
   - *gfortran-lib* 
   - *blas-lib*
   - *lapack-lib*

   You need to check that your dynamic library path has been properly
   set up in the shell, by setting up your .bashrc (or equivalent
   shell init file, or equivalent environment variable settings).

*** TODO [#A] Setting up BLAS/LAPACK on Microsoft OS

    i.e. compilation environment and tools, if needed, on Microsoft Windows?

    For windows, we recommend you use cygwin to get straightforward
    access. I'll document the steps if there is a demand.

*** Linux

    Get lapack-dev, blas-dev from your Linux distribution.

    i.e., something like:

#+BEGIN_SRC shell
sudo apt-get install libblas
sudo apt-get install liblapack
#+END_SRC

    For Linux, if installed in a weird place, you need to make sure
    the loader looks for it.

#+begin_src shell
set LD_LIBRARY_PATH=$LD_LIBRARY_PATH:???
#+end_src

    and need to replace the ???

    SBCL is known to work (0.58+, 1.1.1+)

    CCL is thought to work

    (CMUCL and CLISP may work, but not tested)

*** TODO MacOSX

   For Mac OSX set

#+BEGIN_SRC sh
   export DYLD_FALLBACK-LIBRARY_PATH=$DYLD_FALLBACK_LIBRARY_PATH:/opt/local/lib
#+END_SRC

    *FIXME*: Tony has no clue, please fill this in since most mailing
    list folks use Macs.

    - CCL is thought to work
    - SBCL is known to work

    For OS X: change the parameters as suggested in the file. Both
    BLAS and LAPACK are pre installed on Mac OSX.

** Install Quicklisp

   LispCabinet has it preinstalled, and you can use that version to upgrade

   Debian/Ubuntu also allow you access to a (possibly outdated)
   version.  Not sure about upgrade potential.

   On Linux and MacOSX, I would recommend using the instructions at
   the QuickLisp www site (http://www.quicklisp.org).

** Install Git

   Unfortunately, as much as we really would like to get rid of this
   PITA stage, we are pre-alpha, and that means no chance, unless you
   want to fix your own bugs and copy/paste fixes, etc.  Much simpler
   to figure out a small bit of git.

   Hopefully, your distribution (Linux) has it, and instructions for
   getting it, along with tutorials and documentation, can be found
   for MacOSX and Microsoft Windows on http://www.git-scm.org/

   GitHub also has a Microsoft Windows application that might be
   useful for fetching and working with GitHub repositories (including
   this one).

*** for mac osx

#+begin_src sh
sudo port install git
#+end_src

*** for linux

    Using Debian, Ubuntu, or other apt-get based distributions as an example:

#+begin_src sh
sudo apt-get install git
#+end_src

** Using git to fetch Common Lisp Statistics

   At this stage, we need to identify where you will put the D/L'd
   package.  If you are have an existing, highly tuned quicklisp
   setup, please figure it out and jump to the next stage, but
   basically you need to pull common-lisp-stat from Blindglobe's
   repository, along with a few others.  

#+begin_src sh
cd ~/quicklisp/local-projects/
git clone https://github.com/blindglobe/common-lisp-stat.git
#+end_src

   Regarding the "few others", all of them are currently part of
   QuickLisp, so you only need them if you want to develop with them.
   Which isn't entirely a bad idea.

   These would include:

| lisp-matrix | CLS         | generic interface to BLAS and LAPACK using matrix like API |
| cl-blapack  | lisp-matrix | BLAS / LAPACK FFI                                          |
| fnv         | lisp-matrix | foriegn-numeric-vectors, C-storage for lisp-matrix         |
| ffa         | lisp-matrix | foriegn-??-arrays, LISP storage for lisp-matrix            |
| lift        | CLS         | unit-testing                                               |
| listoflist  | CLS         | list as an array data structure                            |
| xarray      |             |                                                            |

** Compile and load dependencies.

Start up your Common Lisp implementation and type in:

#+begin_src lisp
  (ql:register-local-projects)
  (ql:quickload :cls)
#+end_src

Retire for a well earned coffee and upon your return you should find
the package completely installed.  Obviously, potential errors can
creep in with spelling the filenames correctly, so be careful.

And now, everything should be working.  This is the case for at least
one person, so data on failures is very welcome.

** Start trying it out

Now, load into your IDE or lisp, the files in the examples directory,
such as:

| 00-loadingData.lisp        |
| 02-DSVloading.lisp         |
| 04-dataManipulation.lisp   |
| 10-basicEDA.lisp           |
| 20-visual-2d-cairo2.lisp   |
| 50-TTestExample.lisp       |
| 60-regressionExamples.lisp |
| linear-regression.lisp     |
| ls-demo.lisp               |
| ls-demo-ls1.lisp           |
| XX-readMe.lisp             |

And more

** Example Usage steps

Load the example lisp files 
  
*** change directory into the CommonLispStat working directory.

This is just for directory convenience, not for any real reason.

#+begin_src sh
cd ~/quicklisp/local-projects
#+end_src

*** start your lisp

#+begin_src sh
sbcl
#+end_src

or 

#+begin_src sh
CCL
#+end_src

*** follow the commands in the *ls-demo.lisp* (need to add link) file, i.e.


#+BEGIN_SRC lisp
 (ql:quickload :cls)

 (in-package :cls)
#+END_SRC 

    Initially we will work in the cls package as all the basic
    functions we would need are present

    For serious work we would create our own workspace and save it in
    a separate package, but for now we will take this short cut.

#+BEGIN_SRC lisp
 (normal-rand 20)
 (setf mytest (normal-rand 20))
#+END_SRC

    and see if they work (basic CFFI functionality for external C
    library, LIFT package for unit-testing framework to ensure run
    time stability).
  

*** DONE Setup a place to work

    In Common Lisp, you need to select and setup namespace to store
    data and functions.  There is a scratch user-package, or sandbox,
    for CLS, *cls-user* , which you can select via:

#+BEGIN_SRC lisp -n :tangle "readme-example.lisp"
(in-package :cls-user)
#+END_SRC

    and this has some basic modules from CLS instantiated (dataframes,
    probability calculus, numerical linear algebra, basic summaries
    (numerical and visual displays).

    However, it can be better is to create a package to work in, which
    pulls in only desired functionality:

#+BEGIN_SRC lisp +n :tangle "readme-example.lisp"
  (in-package :cl-user)
  (defpackage :my-package-user
    (:documentation "demo of how to put serious work should be placed in
      a similar package elsewhere for reproducibility.  This hints as to
      what needs to be done for a user- or analysis-package.")
    (:nicknames :my-clswork-user)
    (:use :common-lisp ; always needed for user playgrounds!
          :lisp-matrix ; we only need the packages that we need...
          :common-lisp-statistics
          :cl-variates
          :lisp-stat-data-examples) ;; this ensures access to a data package
    (:shadowing-import-from :lisp-stat
        ;; This is needed temporarily until we resolve the dependency and call structure. 
        call-method call-next-method
  
        expt + - * / ** mod rem abs 1+ 1- log exp sqrt sin cos tan
        asin acos atan sinh cosh tanh asinh acosh atanh float random
        truncate floor ceiling round minusp zerop plusp evenp oddp 
        < <= = /= >= > > ;; complex
        conjugate realpart imagpart phase
        min max logand logior logxor lognot ffloor fceiling
        ftruncate fround signum cis
  
        <= float imagpart)
  
    (:export summarize-data summarize-results this-data this-report))
  
  (in-package :my-clswork-user) ;; or :my-package-user
  
  (setf my-data
        (let ((var1 )) ))
#+END_SRC

    We need to pull in the packages with data or functions that we
    need; just because the data/function is pulled in by another
    package, in that package's namespace, does NOT mean it is
    available in this name space.  However, the
    *common-lisp-statistics* package will ensure that fundamental
    objects and functions are always available.

*** TODO Get to work [0/3]
**** TODO Pull in or create data

**** TODO Summarize results

**** TODO Save work and results for knowledge building and reuse 

One can build a package, or save an image (CL implementation
dependent), or save text files.
  
*** TODO Inform  moi of problems or successes

    mailto:blindglobe@gmail.com if there is anything wrong, or
    even if something happens to work.

    Current beliefs:
    - SBCL is target platform.   CCL and CMUCL should be similar.
    - CLISP is finicky regarding the problems that we have with CFFI
      conversation.  In particular that we can not really do typing
      that we need to take care of.  I think this is my (Tony's)
      problem, not someone elses, and specifically, not CLISP's
    - Need to test ECL.
      
* Introduction
** Core Philosophy

  "Languages shape how we ..."   Need to get and insert this quote
  that Duncan Temple-Lang found.

  The API should distinguish between the realization and the
  statistical interpretation.  Goal is to teach statisticians how to
  think "systems-computationally", and programmers, comp-sci types,
  informaticists and other "data scientists" how to think
  "statistically", in order to get a jump on the competition.

  The goal of this system is to promote a change in thinking, to move
  the data analysis approach, currently stuck in a mix of 70s-early
  90s approaches, into a new generation/level.

** Design Philosophy

   The approach we are taking is one where we provide a general
   method, and some fundamental building blocks, but don't force users
   into approaches in order to allow for experimentation.

   DSL's should be built on top of the core packages, as needed or
   wanted.  

   (TonyR:)  The DSL I want to build is a verbose statistically
   precise computing language, but we need quality code underneathe
   (which others could use for specialized terse DSL's).

   DSL: domain specific language.

* History

   See files in file:Doc/  for history, design considerations, and
   random, sometimes false and misleading, musings.

** XLispStat


*** ViSta

*** ARC

** Common LispStat

   Initial development, 1989 time frame, partially developed during a
   visit by Luke Tierney to Bell Labs.

** Common Lisp Statistics

   This system.

* Local modifications, Development, Contributions

  Since this project is 

#+begin_src shell
git clone git://github.com/blindglobe/common-lisp-stat.git 
cd common-lisp-stat
#+end_src

   will pull the whole repository, and create a "master" branch to
   work on.  If you are making edits, which I'd like, you don't want
   to use the master branch, but more to use a topic-centric branch,
   so you might:

#+begin_src shell
git checkout -b myTopicBranch
#+end_src

and then work on myTopicBranch, pulling back to the master branch when
needed by

#+begin_src shell
git checkout master
git pull . myTopicBranch
#+end_src

(or
#+begin_src shell
git rebase myTopicBranch
#+end_src
)

BETTER DOCUMENTATION EXAMPLES EXIST ON-LINE (on the git WWW site
mentioned above)!! PLEASE READ THEM, THE ABOVE IS SPARSE AND MIGHT BE
OUTDATED!

** Contributing through GitHub

   Alternatively, one can work on the github repositories as well.
   They are a bit differently organized, and require one to get a
   github account and work from there.

   basically, fork the repository on github on the WWW interface, then
   make a branch (as below), push back the branch to github, and
   notify the main repository that there is something to be pulled.
   And we'll pull it back in.

** Commiting with the MOB on repo.or.cz

of course, perhaps you want to contribute to the mob branch.   For
that, after cloning the repository as above, you would:

#+begin_src shell
    git checkout -b mob remotes/origin/mob
#+end_src

(work, work, work... through a cycle of

#+begin_src shell
         
	 git add 
	 git commit -m "what I just did"
#+end_src

 ad-nauseum.  When ready to commit, then just:

#+begin_src shell
     git push git+ssh://mob@repo.or.cz/srv/git/CommonLispStat.git mob:mob
#+end_src

)

and it'll be put on the mob branch, as a proposal for merging. 

Another approach would be to pull from the topic branch into the mob
branch before uploading.   Will work on a formal example soon.

(the basic principle is that instead of the edit cycle on mob, do
something like:

#+begin_src shell
  git checkout mob
  git pull . myTopicBranch   
  git push git+ssh://mob@repo.or.cz/srv/git/CommonLispStat.git mob:mob
#+end_src

)

** Licensing

   We currently are using and recommend the MIT style license approach.

* Footnotes

[fn:1] I´m not including instructions for Emacs or git, as the former
is dealt with other places and the latter was required for you to get
this.  Since disk space is cheap, I´m intentionally forcing git to be
part of this system.  Sorry if you hate it.  Org-mode, org-babel, and
org-babel-lisp, and hypo are useful for making this file a literate
and interactively executable piece of work. 

本源码包内暂不包含可直接显示的源代码文件,请下载源码包。