Merge branch 'master' into gh-pages

Conflicts:
	.gitignore

2 files changed, 345 insertions, 97 deletions

diff --git a/docs/TODO b/docs/TODO
index 0d5d28b..68c5393 100644
--- a/docs/TODO
+++ b/docs/TODO
@@ -1,18 +1,50 @@
 All:
-    - multi-line read
-    - loop/recur ?
-    - hash-maps with non-string keys
-    - hash-map with space in key string (make)
-    - keyword type
-    - gensym reader inside quasiquote
-
-    - per impl tests for step5_tco, step9_interop (if possible)
+    - Finish guide.md
+
+    - test to check args set properly
+    - test to make sure slurp captures final newline
+    - make sure errors propagate/print properly when self-hosted
+
+    * change perf test to run for 10 seconds and then calculate number
+      of iterations per second
+    - redefine (defmacro!) as (def! (macro*))
+    - runtest expect fixes:
+        * stop using expect, so we can drop --raw option
+        * fix C#, VB
+        - fix long line splitting in runtest
     - regular expression matching in runtest
+    - add re (use in rep) everywhere and use that (to avoid printing)
+    - Implement/fix interop
     - Print full exception when test gets EOF from expect
 
+    - metadata on symbols
+    - metadata as a map only. ^ merges metadata in the reader itself.
+      Line numbers in metadata from reader.
+    - protocols!
+        - https://github.com/pixie-lang/pixie
+        - http://www.toccata.io/2015/01/Mapping/
+    - namespaces
+        - environments first class: *ENV*, *outer* defined by env-new
+        - namespaces is *namespaces* map in environment which maps namespace
+          names to other environments.
+        - def! become an alias for (env-set! *ENV* 'sym value)
+        - Namespace lookup: go up the environment hierarchy until
+          a *namespaces* map is found with the namespace name being
+          looked up. Then the symbol would be looked up starting in
+          the namespace environment. Need protocols first probably.
     - Break out impl eval into step0.5
     - Fix quasiquoting of vectors
 
+    - Get self-host working at earlier step:
+        - Move try* to step6
+        - Remove macros from mal
+
+    - multi-line REPL read
+    - loop/recur ?
+    - gensym reader inside quasiquote
+
+    - per impl tests for step5_tco
+
 ---------------------------------------------
 
 Bash:
@@ -21,112 +53,139 @@ Bash:
 
 C:
     - come up with better way to do 20 vararg code
-    - GC
+    - GC: use http://www.hboehm.info/gc/
+    - fix mal/clojurewest2014.mal
 
 C#:
-    - step9_interop
+    - fix command line arg processing (doesn't run file specified)
+    - accumulates line breaks with mal/clojurewest2014.mal
+    - step9_interop:
+      http://www.ckode.dk/programming/eval-in-c-yes-its-possible/
 
 Clojure:
+    - make indent consistent across steps (e.g. step5, step8)
+    - fix mal/clojurewest2014.mal
+
+CoffeeScript:
+    - make target to compile to JS
+    - fix "user> " prompt with mal/clojurewest2014.mal
+
+Go:
+    - consider variable arguments in places where it makes sense
+        https://gobyexample.com/variadic-functions
+
+Haskell:
+    - TCO using seq/bang patterns:
+      http://stackoverflow.com/questions/9149183/tail-optimization-guarantee-loop-encoding-in-haskell
+    - immediately exits mal/clojurewest2014.mal
 
 Java:
-    - step9_interop
+    - Use gradle instead of mvn
+        http://blog.paralleluniverse.co/2014/05/01/modern-java/
+    - MAL formatting is a bit off with mal/clojurewest2014.mal
 
 Javascript:
+    - interop: adopt techniques from miniMAL
+    - fix "user> " prompt with mal/clojurewest2014.mal
+
+Lua:
+    - time-ms should get actual milliseconds
 
 Make:
     - allow '_' in make variable names
+    - hash-map with space in key string
+    - Fix: make -f stepA_mal.mk ../mal/step6_file.mal
+        (slurp "../tests/incA.mal")
+        (read-string "(+ 2 3)")
     - errors should propagate up from within load-file
+    - GC: expore using "undefined" directive in Make 3.82
 
 Mal:
     - line numbers in errors
     - step5_tco
-    - step9_interop
+
+miniMAL:
+    - figure out why {} literals are "static"/persistent
 
 Perl:
+    - fix metadata on native functions
+    - implement conj
 
 PHP:
+    - formatting messed up with mal/clojurewest2014.mal
 
 Postscript:
-    - negative numbers
+    - add negative numbers
+    - fix blank line after comments
+    - fix command line arg processing (doesn't run file specified)
+    - formatting messed up with mal/clojurewest2014.mal
 
 Python:
-    - error: python ../python/stepA_more.py ../mal/stepA_more.mal ../mal/stepA_more.mal
+    - error: python ../python/stepA_mal.py ../mal/stepA_mal.mal ../mal/stepA_mal.mal
+    - interop tests
+
+R:
+    - tracebacks in errors
+    - fix running from different directory
+    - formatting messed up with mal/clojurewest2014.mal
+
+Racket
+    - metadata on collections
 
 Ruby:
 
+Rust:
+    - use built-in regex once fixed:
+        https://github.com/rust-lang/rust/issues/18034
+        https://github.com/rust-lang/rust/issues/18035
+    - fix 'make all' invocation of cargo build
+    - formatting messed up with mal/clojurewest2014.mal
+
+Scala
+    - readline
+    - fix exception when finished running something on command line
+
+VB.Net
+    - convert readline.cs to readline.vb
+
 
 ---------------------------------------------
 
 
 Future Implementations:
 
-    - Rust:
-        - http://www.rustforrubyists.com/book/index.html
-        - http://static.rust-lang.org/doc/0.9/complement-cheatsheet.html
-        - http://pzol.github.io/getting_rusty/
-        - release notes:
-            - https://github.com/mozilla/rust/wiki/Doc-detailed-release-notes
-        - this week in rust:
-            - http://cmr.github.io/
-        - readline:
-            - http://redbrain.co.uk/2013/11/09/rust-and-readline-c-ffi/
-            - http://www.reddit.com/r/rust/comments/1q9pqc/rust_cffi_and_readline/
-            - https://github.com/dbp/rustrepl
-        - hash-map:
-            - http://static.rust-lang.org/doc/master/std/hashmap/index.html
-            - http://static.rust-lang.org/doc/master/std/hashmap/struct.HashMap.html
-        - vector/list:
-            - http://static.rust-lang.org/doc/master/std/vec/index.html
-        - example code:
-            - https://github.com/dradtke/rust-dominion/blob/master/dominion/mod.rs
-
-    - Redmonk languages from Jan 2014:
-        http://sogrady-media.redmonk.com/sogrady/files/2014/01/lang-rank-114-wm.png
-    
-        - Tier 1
-            * JavaScript
-            * Java
-            * PHP
-            * C#
-            * Python
-            - C++
-            * Ruby
-            * C
-            - Objective-C
-            * Perl 
-            * Shell (Bash 4)
-
-        - Tier 2
-            - Scala
-            - Haskell
-            * Clojure
-            - CoffeeScript
-            - Visual Basic
-            - Groovy
-            - Go
-            - Lua
-            - Erlang
-            - Emacs Lisp
-            - Assembly
-            - Scheme
-            - FORTRAN
-            - Dart
-            - F#
-            - D
-
-        - Tier 3
-            - TypeScript
-            - Racket
-            - HaXe
-            - Pascal
-            - VimL
-                - https://github.com/tpope/timl
-            - Common Lisp
-            - Rust
-            - M (OpenM/MUMPS)
-            - Factor (Stack-based)
-
-    - Others:
-        - Forth (Stack-based)
-        - BF (Crazy)
+    - Ada (gnat)
+        - http://rosettacode.org/wiki/Regular_expressions#Ada
+
+    - C++
+
+    - Groovy
+        - http://groovy-lang.org/learn.html
+        - http://groovy-lang.org/structure.html
+
+    - Erlang
+
+    - F#
+
+    - Haxe
+        - http://api.haxe.org/
+        - http://haxe.us/haxe_tutorial.html
+
+    - Julia
+
+    - Nim
+
+    - Objective-C:
+
+    - Pascal:
+        sudo aptitude install fp-compiler-2.6.2
+
+    - VimL
+        - https://github.com/tpope/timl
+
+    - Tcl
 
+    - TeX/LaTeX
+        - Basic interpreter in TeX: http://ctanhg.scharrer-online.de/pkg/basix.html
+        - Cheat Sheet: http://www.stdout.org/~winston/latex/latexsheet.pd
+        - latex '\nonstopmode\input' blah.tex
diff --git a/docs/step_notes.txt b/docs/step_notes.txt
index 09bc356..e28761a 100644
--- a/docs/step_notes.txt
+++ b/docs/step_notes.txt
@@ -4,6 +4,19 @@ Step Notes:
     - prompt, input, READ, EVAL, PRINT, output
     - readline module
         - display prompt, read line of input
+    - Details:
+        - get your language compiler/interpreter running
+        - create step0_repl.EXT
+            - loop that reads input, calls rep, writes output, exits
+              on EOF/Ctrl-D
+            - rep calls PRINT(EVAL(READ(str)))
+            - READ, EVAL, PRINT just return input parameter
+        - modify toplevel Makefile
+            - add language (directory name) to IMPLS
+            - add <lang>_STEP_TO_PROG entry
+            - add <lang>_RUNSTEP entry
+        - for a compiled language, add <lang>/Makefile
+            - targets: all, step*, stats, stats-lisp, 
 
 - use native eval in EVAL if available
 
@@ -17,6 +30,7 @@ Step Notes:
     - types module:
         - add boxed types if no language equivalent:
             - nil, true, false, symbol, integer, string, list
+        - error types if necessary
     - reader module:
         - stateful reader object
             - alternative: mutate token list
@@ -33,15 +47,42 @@ Step Notes:
         - read_atom (not atom type)
             - return scalar boxed type:
                 - nil, true, false, symbol, integer, string
+            - skip unquoting
     - printer module:
         - _pr_str:
             - stringify boxed types to their Mal representations
             - list/array is recursive
+            - skip quoting
     - repl loop
         - catch errors, print them and continue
         - impls without exception handling will need to have a global
           variable with checks for it at the beginning of critical
           code sections
+    - Details:
+        - copy step0_repl.EXT to step1_read_print.EXT
+            - modify Makefile if compiled
+            - call reader.read_str from READ
+            - pass through type returned from read_str through
+              READ/EVAL/PRINT
+        - create reader.EXT
+            - if regex support (much easier) then tokenize with this:
+                /[\s,]*(~@|[\[\]{}()'`~^@]|"(?:\\.|[^\\"])*"|;.*|[^\s\[\]{}('"`,;)]*)/g
+            - add read_str:
+                - call tokenize
+                - handle blank line (exceptions, return code, global
+                  depending on lang features)
+            - read_str -> read_form -> {read_list, read_atom}
+                - mutable reader thing
+        - create printer.EXT
+            - _pr_str function which basically reverses read_str and
+              returns a string representation
+        - run `make test^EXT^step1`. Much of the basics should pass up
+          to vectors
+        - implement read_hash_map (can refactor read_list)
+        - import read_vector 
+            - probably want to define types for List and Vector in
+              types.EXT that extend or wrap native arrays
+        - run `make test^EXT^step1`. All mandatory should pass
 
 - comments
 
@@ -82,8 +123,23 @@ Step Notes:
     - EVAL/apply:
         - if not a list, call eval_ast
         - otherwise, apply first item to eval_ast of (rest ast)
-    - repl_env as simple one level assoc. array (or hash_map)
+    - repl_env as simple one level hash map (assoc. array)
         - store function as hash_map value
+    - Details:
+        - copy step1_read_print.EXT to step2_eval.EXT
+        - create repl_env hash_map) with +, -, *, /
+            - store anon func as values if possible
+        - types.EXT
+            - implement symbol? (symbol_Q) and list? (list_Q)
+        - add env param to EVAL and add to rep EVAL call
+        - EVAL
+            - if not list call eval_ast
+            - otherwise eval_ast, and call first arg with rest
+        - eval_ast
+            - if symbol?, lookup in env
+            - if List, EVAL each and return eval's list
+            - otherwise, return original
+        - optional: handle vector and hash-map in eval_ast
 
 - vectors
     - eval each item, return new vector
@@ -99,6 +155,22 @@ Step Notes:
     - EVAL/apply:
         - def! - mutate current environment
         - let* - create new environment with bindings
+    - Details:
+        - cp step2_eval.EXT to step3_env.EXT
+        - add env.EXT if lang support file dep cycles, otherwise, add
+          to types.EXT
+            - Env type
+            - find, get, set methods/functions
+        - use Env type instead of map/assoc. array
+        - eval_ast: use method for lookup
+        - EVAL:
+            - switch on first symbol
+                - def!
+                    - set env[a1] to EVAL(a2, env)
+                - let*
+                    - loop through let building up let_env
+                    - EVAL(a2, let_env)
+                - move apply to default
 
 - step4_if_fn_do
     - types module:
@@ -126,13 +198,45 @@ Step Notes:
             - otherwise needs a way of representing functions that can
               have associated metadata
     - define "not" using REP/RE
+    - Details:
+        - cp step3_env.EXT to step4_env.EXT
+            - modify Makefile if compiled
+        - env.EXT
+            - add binds and exprs args. Create new environments with
+              exprs bound to binds. If & symbol, bind rest of exprs to
+              next bind symbol
+        - EVAL:
+            - do:
+                - eval_ast [1:], then return last eval'd element
+            - if
+                - EVAL(a1)
+                    - if true EVAL(a2)
+                    - else EVAL(a3), unless no a3 then return nil
+            - fn*
+                - if available use function closures to return a new
+                  native function that calls EVAL(a2, Env(env, a1, fargs))
+                - otherwise, store exp, params and env in a structure
+        - core.EXT
+            - create ns object to hold core namespace
+            - move numeric operators here
+            - add comparison operators
+            - add list, list?, empty?, count
+        - run make test^EXT^step4
+        - implement equal?/equal_Q in types.EXT and refer in core.ns
+        - implement not as rep("(def! not (fn* (a) (if a false true)))")
+        - run make test^EXT^step4: should pass everything except
+          string routines
+        - implement: pr-str, str, prn, println in core.EXT and
+          refer in core.ns
+            - should leverage pr-str from printer.EXT
+            - add reader/printer string quote/unquote
 
 - step5_tco
     - types module:
         - mal function type:
-            - stores: func, exp, env, params
-            - func is EVAL in native mal case, otherwise reference to
-              platform function
+            - stores: eval, exp, env, params
+            - eval is EVAL in native mal case (needed for map function
+              later), otherwise reference to platform function
             - if metadata support, then store exp, env, params as
               metadata
     - printer
@@ -142,8 +246,24 @@ Step Notes:
         - cases where we directly return result of EVAL, instead set
           ast and env to what would be put in the EVAL, then loop.
             - do, if, "apply"
-            - for "apply" case, set env to new Env based on properties
-              on the function
+            - "apply"
+                - if mal function type
+                    - set env to new Env based on properties on the function
+                - if native function, same as before
+    - Details:
+        - types.EXT
+            - create Mal function type to store eval, exp, env, params
+        - cp step4_if_fn_do.EXT to step5_tco.EXT
+            - wrap EVAL in infinite while loop
+            - in let*, do, and if:
+                - set ast and env and loop (no return)
+            - in fn* create Mal function type
+        - if compiled, update Makefile
+        - in apply, test if Mal function type:
+            - if so, generate new env from stored env, args and callee
+              params
+            - set ast to stored ast
+
 
 - step6_file
     - core module:
@@ -152,6 +272,26 @@ Step Notes:
     - set *ARGV*
     - if files on command line, use load-file to run first argument
       using rest as arguments
+    - Details:
+        - cp step5_tco.EXT to step6_file.EXT
+        - if compiled update Makefile
+        - add eval to repl_env
+            - if no (or limited closures) may have to add an "eval"
+              case to EVAL and use function which gets root of
+              environment to env.EXT (see rust).
+        - add empty *ARGV* list to repl_env
+        - in core.ns:
+            - wrap printer.read-str as read-string
+            - implement slurp
+        - implement load-file using rep
+        - test:
+            (load-file "../tests/inc.mal")
+            (inc3 10)
+        - implement command line execution
+        - test:
+            ./step6_file ../tests/incA.mal
+            =>9
+        - implement comments in reader.EXT (ignore in tokenize)
 
 - step7_quote
     - add is_pair and quasiquote functions
@@ -164,6 +304,15 @@ Step Notes:
     - reader module:
         - add reader macros to read_form for quote, unquote,
           splice-unquote and quasiquote
+    - Details:
+        - cp step6_file.EXT to step6_quote.EXT
+        - if compiled update Makefile
+        - implement reader macros (', `, ~, ~@) in reader
+        - retest make test^go^step1
+        - add is_pair and quasiquote
+        - add quote and quasiquote cases to EVAL
+        - implement cons and concat in core.EXT
+        - retest test^go^step7
 
 - step8_macros
     - types
@@ -178,12 +327,19 @@ Step Notes:
     - EVAL:
         - add 'defmacro!' and 'macroexpand'
         - set ismacro property on function
+    - Details:
+        - cp step7_quote.EXT to step8_macros.EXT
+        - if compiled update Makefile
+        - add isMacro property to Mal Function type
+            - may need to go back and adjust step5-7
+        - implement is_macro_call and macroexpand
+        - call macroexpand on ast before apply in EVAL
+        - add defmacro! and macroexpand to EVAL switch
+        - make test^go^step8 should pass some basic macros
+        - add nth, first, and rest to core.ns
+        - make test^go^step8 should now pass
 
-- step9_interop
-    - convert returned data to mal data
-        - recursive, similar to pr_str
-
-- stepA_more
+- step9_try
     - core module:
         - throw function
         - apply, map functions: should not directly call EVAL, which
@@ -195,6 +351,33 @@ Step Notes:
           otherwise extracts full value
     - set and print *host-language*
     - define cond and or macros using REP/RE
+    - Details:
+        - cp step8_macros.EXT to stepA_try.EXT
+        - if compiled update Makefile
+        - core.ns implement nil?, true?, false?, symbol?, sequential?,
+          vector, vector?
+        - add mal error type which wraps normal mal type
+            - in core.ns add throw which wraps type in mal error type
+              and throws/raises/sets exception
+        - add try*/catch* support to EVAL
+            - if mal error type, bind to catch* bind symbol
+            - otherwise, bind string of error to catch* bind symbol
+        - implement apply, map in core.ns
+        - make test^go^stepA
+        - implement readline.EXT
+            - provide option (e.g. commented out) to link with GNU
+              readline (GPL) or libedit (BSD)
+        - add hash-map functions: hash-map, map?, assoc, dissoc, get,
+          contains?, keys, vals
+        - add metadata support to List, Vector, HashMap, and Functions
+            - add reader macro
+            - may need to box HashMap and native functions
+        - add atom type, reader macro and functions: with_meta, meta
+        - get `make test^go^stepA` to fully pass
+        - get `./stepA_try ../mal/step1_read_print` to pass
+            - continue for each mal step until ../mal/stepA_try
+            - Now self-hosting!
+
 
 - Extra defintions needed for self-hosting
     - core module:
@@ -202,9 +385,6 @@ Step Notes:
         - vector, vector?
 
 
-- Other misc:
-    - conj function
-
 - atoms
     - reader module:
         - @a reader macro -> (deref a)
@@ -221,3 +401,12 @@ Step Notes:
             - clone/copy of collections
     - core module:
         - add with-meta, meta functions
+
+- Other misc:
+    - conj function
+
+- stepA_mal
+    - convert returned data to mal data
+        - recursive, similar to pr_str
+    - Details:
+