TO DO FOR PARSELY:

(B)ug  
(D)esign 
(E)ssential Feature
(F)eature 
Code (O)rganization
(P)roof of concept
(L)ow-priority

Pac(K)aging
Do(C)umantation

X?= Same as X, but I'm not sure if I did it the right way.

- Not done
. half done
o done
X Unresolved/discarded
D deferred

For version 0.0.1:  [Respectable, but without a couple of needed features.]
	o Get states working. (F)
	o Make exceptions reasonable. (B)
	o Make Start declaration. (E)
	o Warn when choosing an alternative at random. (F)
	o Figure out node replacement. (D)
	o Save initial space. (B)
	o Give better errors for:
		o duplicate sequence tags (E)
	o Add 'insert' functions. (E)
	o Add slice, del functions. (E)
	o Add 'set' functions. (E)
	oXMake 'OPT' have a proper set function. (E)
	o Have Multi, Seq, and Opt nodes be sep. classes? (O)
	o Sane defaults for space. (F)
	o Make $ and ^ work properly. (F)
	o Add line numbers to tokens.

For version 0.0.2: [Indices work, and stuff is more flexible.]	
	o No-space states. (F)
        o Create default nodes of a given kind. (F)
	  o Leaves
	  o Rules, to one level.
	  D Rules, in general
	  	  This creates problems with recursivity.  Better to require
	    	  values, or to go only one level deep?  Let's see what we 
		  need.
	o RE: Deal with different kinds of groups properly. (E)	
	o RE: Make nongreedy matches work. (E)
	o Make toplevel indices work, read-only. (F)
	o Make relative mappings work, read-only. (F)
	o Add way to specify:
		D debug, verbose (F)
		o type of scanner, parser. (D)
		o dotall option (D)
		o nocase option (D)
		o options in general. (D)
	o Get semantics for assignment right! The below must be defined
	  for all nodes type, including leaves. (D)
		o What does .set do? (D)
		o What does [x]= do? (D)
		o What does .x= do? (D)
	o Better regression tests for all of the above. (E)
		o Testing framework
		o remanip
			o remanip valid cases
			o X-specified patterns.
		o lexing
			o pcre mode with patterns, lits,I,X, etc..
			o pcre mode, casei, with patterns, lits, I,X, etc.
			o pcre mode with nospace, states (x and i)
		o parsing
			o Grammar with opt, seq, multi(+,*,N,Sep), alt.
		o nodes
			o Set methods for all nodes.
			o Try to do [x], .x for all node types (no indexing)
			o Insert/delete/append slices/lists on multi nodes
			o NewNode, in all its glory. 
			  {type/notype} x {format/noformat}

For version 0.0.2b: [Design issues found during error testing.]
	o Remove Opt node. (O)
	o Regularize set operations. (B)
	o Add sensible string coercions. (E)
	o Maybe replace all the cruft in file with __getattr__. 
	o New space handling system:  'None' signifies default space.
	o Only escape characters for sexprs headed for verbose dumps.
	o String coersion on leaves? (D)

For version 0.0.3: [Stable, usable, respectable.  Code is clean. 
                    But indexing still doesn't work.]
	o RE: Make all the rest of perl-style regexen work. (F)
	o Add reasonable doc strings. (O)
		o Format
		o Tree
		o Remanip
		o Scanner
		o Parser
	o Break parsely into parsely.tree/parsely.format/parsely.parse. (O)
	o Eliminate braindamaged dual-duty for Leaf-as-token and 
	  Leaf-as-value. (O)
	o Remove old, broken indexing cruft. (O)
	o Separate types and rules? (OD)
		o No, but remove type-type-switches.
	o Better error handling 
		o Better error handling on bad tokens. (F)
		o Errors for malformed regexen, unusable perlism, etc. (F)
		o Better errors on forgetting RE/LIT/SEQ/etc. (F)
	o State-sensitive space. (F)
	o Make interface to parser be better. (F)
	o REManip Visitor to test for perlism (F)
	o Implement __repr__. (F)
	o Complement sep with term (F).
	o Provide node methods to get sexpr. (F)
	o Comment code, especially in tree, format modules. (O)
	o Search for places where I should clone lists on input/output.
	o New regression tests:		
		o Nospace
		o State-sensitive space
		o Repr
		o Terminated lists
		o sexpr(1,2,3)
		o Perlism detector
		o Two tokens with same name.
		o Error handling
			o On remanip
			o On grammar
				o Forgetting RE/LIT/SEQ/etc.
				o Nonexistant tokens/types/states/patterns
				o No start symbol
				o Duplicate declarations
			o On lexing
				o No valid token matches.
			o On parsing
				o Doesn't match.

For version 0.0.4: [Indexing]
	o Clear up remaining indexing issues:
		D Can you do foo['bar'] = baz if foo['bar'] exists? (D)
		D Can you do foo['bar'] = baz if foo['bar'] DNE? (D)
		D Can you do del foo['bar']? (D)
		D How can you do foo.has_key('bar')? (DE)
		o Implement 'paths':  index('member.sub') (F)
		D Can/should we reindex automatically? (D)
		D Can/should we index someplace other than "1 level up?" (D)
		D Provide means to display indexing info? (D)
		o Define all semantics.
	o Implement paths:  foo.get_path("member.sub.3.sub2") (F)
	o Minimal traversal function. (F)
	o Design buildindex function. (D)
	o Implement 'buildindex' function for tree.  Semantics should
	  be minimal;  type, location-path, sub-path. (F)
		o Give means to display indexing info as it's generated. (F)
	o Implement Visitor for tree.  Should roughly follow acyclic visitor
	  pattern. (F)
	o Remove extraneous node methods (O)
	o Make nodes give proper exceptions. (E)
	o New regression tests:
		o getpath
		o setpath
		o traverse
		o indexing 
		o visitor
		o Errors on:
			o Node access
				o Out of bounds
				o Out of bounds (set)
				o No such attr
				o Bad index type
				o Insert badly typed node
		
For version 0.0.5: [Actions and includes are supported.]
	o Actions. (F)	
		o Scanner actions. (F)
		o Parser actions. (F)
	o Reinterpret. (F)
	o Make space and tokens get handled homogenously by scanner (B)
	o Make space have actions, state transitions &tc, just like tokens. (B)
	o Regression tests for new stuff:
		o Scanner actions
			o CONTEXT
			o MATCH, TYPE, LEN, LINE
			o REINTERPRET
			o SET_TYPE
			o MORE, REST, ACCUMULATE, LESS
			o BEGIN
			o ERROR
		o Parser actions
			o CONTEXT, MATCH, VAL*, LINE
			o ERROR
		o enterState on space.
		
For version 0.0.6: [One more cleanup.  Fix issues found in 0.0.5.  
                    Almost ready to move away from core Python code until the
                    Great C Port.]
	o Need a 'REJECT' action?? (D)
		o No, but we need a SPACE action. (F)
	o Need a 'DONE' action? (D)
		o Call it 'CONTINUE'. (F)
	o Strip initial space from actions in python. (E)
	o Add gettype to nodes. (F)
	o _Really_ separate types from productions. (O)
	o Un-generify translators. (O)
	o Move actions from 'Rule' to SEQ/MULTI/OPT
	o Doc strings, again. (O)
		o In __init__
		o In _util
		o In earley_parser
		o In format
		o In pcre_scanner
		o In pyTrans
		o In remanip
		o In tree
	o Document fields, 'friend' relations. (O)
		o In __init__
		o In _util
		o In earley_parser
		o In format
		o In pcre_scanner
		o In pyTrans
		o In remanip
		o In tree
	o Refactor long methods (O)
	o Rename confusing classes/methods/fields/args/locals. (O)
	o Replace field accesses with getmethods. (O)
		o In format 
		o In _util
		o In earley_parser (_Token, Format, Lexeme.the_action, and 
				    Rule are friend.)
		o In grammar 
		D In pcre_scanner (Format, _Token, Lexeme, remanip.write(B)are 
				   friend.)
		o In __init__
		o In browseTree
		o In pyTrans
		X In remanip (The stuff here is very well encapsulated
			within the package, so I won't worry too much.)
		o In tree
	o Remove state from pcre_scanner and earley_parser objects to make
	  them reentrant. (B)
		o In earley_parser:
			D Context.
			o Circularity.
		o In pcre_scanner: 
			o reinterprets
			o errorCount		
	D Do MDDs, maybe OMs. (O?)
	o Fix all 'xxxx's that I can.   (O)
		o In __init__
		o In _util
		D.In earley_parser
		o In format
		o In pcre_scanner
		o In pyTrans
		o In remanip
		o In tree
	D What goes into parsely.__init__? (O)
	D Parser actions on MULTI should only happen once. (DE)
	o Missing tests:
		o ALT's with unlabeled alternatives. 
		o Patterns with \b
	o Tests for new features:
		o Spaces before actions in python.
		o CONTINUE
		o SPACE
		D Parser actions on MULTI

For version 0.0.7: [Includes, implicit SEP.]
	o Include new spark from latest decompyle.  With any luck,
	  it should decouple earley_parser from spark's internals. (O)
	o Includes (F)
		o Design them first! (D)
			D Includes that compose part of a list
			D Including the same file more than once.
		o IncludeNodes are reasonable in tree. (F)
		o Include logic for parser. (F)
			o Parsing from a non-standard start. (E)
		o Include logic for scanner. (E)
		o Include actions. (E)
			o For parser
			o For scanner
	o Implicit separator/terminators. (F)
		o Extend declarations in format. 
		o Extend typesystem.
		o Add new subtype for _BranchType.
		o Add 'XList' node type.
		o Search for all explicit references to ListNode[Type];
		  maybe replace.
		o Make parser generate properly.
	o When newNode guesses a type variant, it should see if more than
	  one is of the right length.
	o More recursion in newNode
	o Make 'dump' work cross-file.	
	o Parsing optimisations 40%
		o Remove checks from BranchNode.__init__ to newNode. (Gain:
	          about 15%)
		o Stop using so much of _Token.__cmp__ and _unwrapNode.
	          (Gain: about 30%).
		X No gain in separating action version non-action paths
		X No gain in separating cases of _BuildListNode
	o Scanning optimizations 48%
		o Use group() and indices more modestly in scanOneLexeme. 10%
		o Use match.regs instead of groups().  This eliminates a lot
		  of unnecessary object creation and manipulation. 27%
		o Instead of bothering with match.regs, call the match method
	          of the pcre code object directly.  This avoids overhead
		  in the match() method, as well as needless object creation.
		  20%.
		X Only 3% gain for modifying pos instead of s; reject.
	o Use pos instead of s[:] in scanner: enable ^ to dtrt.
	o Regression tests
		o Better driver logic -- remove extra crud.
		o Exclusive sep
		o Exclusive term
		o Improved defaultNode facilities.
		o NewNode guessing proper structnode type.

For version 0.0.8: [Ansillary tools]
	o GTK+ tree browser, in Python. (F)
		o Basic functionality
			o Test it
		o File-dumping
	o Pretty-print s-expressions. (F?)
	o More sophisticated writeout methods (These should be orthogonal).
		o For file output
		o For multi file
	o 'Grammar-style' file-format specification. (E)
		o Basic support (E)
		o Python-code interpolation (F)
		o Format pickling (E)
		o <<EOL form, or {{{ form, or some multiline quoting. (E)
		o /regexen/, m/regexen/, q[strings]. (F)
		o Bison-style 1-char and string tokens in 'grammar.py'
		o State-sensitive space (F)
		o Macros.  (In python: should take strings/regexen/ids, 
		  return strings.) (F)
			o Maybe allow macros with ID/Regex arguments.
			o Allow 'def' in addition to 'defmacro'
		o Exclusive seps.
		o Clean up the scanner code; it's ugly as can be. (O)
		o Decent error handling 
		o Import mechanism. (DE)
		o Actions (F)
	o Add GPL/LGPL notices, copyright notices to all files.
	o Fix all the XXXX's
	o Regression tests:
		o Valid grammars
			o A simple grammar
			o All the features from the grammars
			  in regular test suite, minus actions.
			o String-typed tokens.
			o Macros
			D Actions
				o Test parsing and scanning actions.
				D Test final context as well.
					D Put off until we make Python
					  actions sane!
			o Imports
			o Options
		o Pretty-printing
		D Fmt pickling
			o Basic operation
			D Correctness of read-in pickled grammars
			D This is buggy, and the returns small.  Fix it later!
		o Make sure we can't break python actions by putting
		  insufficiently indented comments in.
		o Try to do scanning without use_pcre_hack.
	o Fix bugs found during testing:
		o Literal strings in grammar need to interpolate \n, \t, etc.
		o Grammar can't handle # in non-x form.  What does perl do?
		o Nonfatal grammar errors should eventually throw exception.
		o Maybe 'COLON separated VALUE' would be better than
		  'sep(VALUE,COLON)'?
		o Make dumpFormat._pat behave properly with 'i' flag.
		o Make dumpFormat._pat behave properly with patterns.
		o The parenthesis around the in-state lists are a pain.

For version 0.0.9:
	o Make python actions object-based instead of the ridiculous
	  ad-hoc scheme we have now.  Design, then implement.
		o Design.
		o Implement in pcre
		o Implement in earley.
		o Strip old cruft from pyTrans; 
		o Shift to new scheme;
		o Adjust tests accordingly.
	o Add first/finally code/actions mechanism
		o Make generated function be a member of Action obj as well.
		o Make scanner cclass generation drive off actions list,
		  as well as token list.
		o Make parser cclass generation drive off actions list,
		  as well as rule list.
		o Add extra field to action.
		o Handle field in context classes
			o Scanner
			o Parser
		o Make grammar capable of generating action kinds.
		o Make compilation drive off actions list.
	o Tests
		o Bad grammars
			o Infrastructure
			o Empty files
			o Runaway token elements.
			o Syntax errors.
			o Missing symbols.
			o Macros, chunks, inline strs, multiline strs, 
			  and line numbers.
			o Missing start symbol.
		o File output
		o Pre/post/fn actions.
			o Pre scanner
			o Pre parser
			o Post scanner
			o Post parser
			o Scanner Fn
			o Parser Fn
		D Includes
			[Includes don't work the way I want them too.
			 It's a bad idea to ship a deliberately broken
			 interface.  Therefore, I'm going to disable them.
			 They're easy enough to fake, anyway -- at
                         least, to fake as well as they are now.  So
                         defer them for 0.2]
			D Parser
			D Scanner
		o Lex an empty file.
	o Bugfixes: Error quality.
		D 'Unrecognized symbol' should say _where_ that symbol
		  occurred.
		o Make all error lines correct
		o Archetecture for error productions, as found to be useful.
		o Macros seem to break line-number checking.
	o parseFile method of format.
	o Outline for documentation

For version 0.1.0: [Alpha-release: Proof of concept, python only]
	o Make use of 'hak_spark' more sane. (O)
		o Build-in an equivalent to the use_pcre_hack test.
		  (That's the big incompatibility.)
		o Provide orig_spark in distro, even if it's not used.
	o Process some of: 
		o ssh_config (PE)
		.Dfstab
	o Rework browseTree (P)
		o Make OO
		o Improve types.
		o Doc fields
		o Improve dump.
	o Regression tests for above. (E)
	o Improve error handling for errors exposed in above process. (E)
	o Documentation. (CE)

For version 0.1.1: [First beta release]
	o Write FAQ and docs.
	- Alpha test
	- Sane build process (automake?) (KE)
	- Go over all 'defered' decisions above.

For version 0.1.2: [Includes done right]
	- Redesign include mechanism
	- Reimplement it.
	- Add tests for include.

For version 0.1.3: [More examples]
	- Regression tests and frontend for ssh.
	- Make tcsh work for me.  Don't make it completely-bulletproof.
	- Make bash work for me.
	- Make procmail work for me.

For version 0.1.4: [Error reporting]
	- Make error messages appear totally sane.... 
          through creative misstatement of their actual origins.
		- When a scan or grammar error is thrown: catch and relay
		  the message.
		- Make errors from actions give useful locations.
		- Make errors from macros give useful locations.
	- Standard macro library.
		- Simple lexemes (F)
		- Regexen (F)
		- Line-based file format (F)

For version 0.2.0: [Stable release]
	- Clever quotes at the bottom of each file. (O)
	- Add automatic 'make profile' as soon as we have real-world tests.
	- Improve profile performance.

For version 0.3.0:  [C support!]
	. Design C API (D)
	o Clean out code generation garbage; 'action context' is a bad idea.
	- Generate flex/pcre/glib scanners, bison/earley parsers
		  code for C (F)
		- Flex scanner (E)
			o Add flex code to remanip
			. Write code to generate flex scanners.
		D Pcre scanner (F)
		D Glib scanner (???)
		- Bison parser (E)
		D Earley parser (F??)
		- Action handling (E)
	. Write C library to manipulate nodes. (DF)
 	- Generate binaries for dynamic/static linking in C (E)
	- Reconsider necessity of pyTrans.
		- Decide whether we really need it.
		- Write rationale in file header.
	- Write regression tests. (E)

For version 0.3.2: [More versitile C]
	- New scanner/parser varieties.
		- Pcre scanner (F)
		- Earley parser (F??)
	- Regression tests

For version 0.3.3: [C++ support]
	- Generate flex/pcre/glib scanners, bison/earley parsers
		  code for C++ (F)
		- Action handling. (F)
		- Namespaces. (E)
	- Write C++ library to manipulate nodes. (DF)
 	- Generate binaries or dynamic/static linking in C++ (F)
	- Write regression tests. (E)

For version 0.3.4: [Python/C integration]
	- Write python wrappers for C library/C parsers. (F)

Later: [Difficulty/value=cost/benefit] out of [6/6]
	- For GTK+ browser:
		- Type browsing [3/3=1]
		- New node, delnode. [3/2=1.5]
		- Indexing. [3/2=1.5]
		- Doc code better. [1/5=.2]
	- GTK+-based grammar generator. [5/2=2.5]
	- Token subfields. [4/5=.8]
	- Perl support. [5/4=1.25]
	- Complex rule declarations in grammar: [5+1/5=1.2] e.g.,  
	     Decl = set? Name "=" Val* Semi
		- This is more complex than it seems.
	- In-line actions in grammar. [4/?=?]
	- Standard libraries for things like <<EOL, C tokens, shell
	  tokens, python tokens, regexen, and so forth. [?/5]
	- Should a singleton struct propagate requests to its member? [3/?]
	- Translate to/from XML; provide an XML schema. [5/3]

Difficulty: 
	1: 1-20 minutes
	2: 20min-1 hour
	3: 1-3 hours
	4: 1 day
	5: more than one day
	6: I have no idea how to do this.
	+1 for hard-to-specify

Value:
	1: Not discernable.
	2: Makes tool slightly more usable, in a way I'm not sure how
	   to take advantage of.
	3: Someone would actually use this.
	4: I think I would use this.
	5: And it would be really handy.
	6: Essential to parsely's usefulness/memetic success.

POSSIBLE NAMES FOR CONF TOOL:
recon
concert(o?)
vulc[oa]n
torc
rlp

