updated docs

project.clj

@@ -12,5 +12,6 @@
   :repl-options {:init-ns propeller.core}
   :jvm-opts ^:replace []
   :plugins [[lein-codox "0.10.8"]]
-  :codox {:output-path "codox"
-          :metadata {:doc/format :markdown}})
+  :codox {:output-path "docs"
+          :metadata {:doc/format :markdown}
+          :doc-paths ["src/docs_src"]})

src/docs_src/A_Guide_To_Propeller.md

@@ -0,0 +1,204 @@
+# A Guide to Propeller
+
+**Propeller**  is an implementation of the Push programming language and the PushGP genetic programming system in Clojure.
+
+For more information on Push and PushGP see http://pushlanguage.org.
+
+## Overview
+
+**Propeller** is a Push-based genetic programming system in Clojure.
+
+<!-- TOC -->
+* [A Guide to Propeller](#a-guide-to-propeller)
+  * [Overview](#overview)
+    * [What can you do with Propeller?](#what-can-you-do-with-propeller)
+  * [Installation](#installation)
+  * [How do I run Propeller on a problem?](#how-do-i-run-propeller-on-a-problem)
+    * [An Example](#an-example)
+    * [Can you use a REPL?](#can-you-use-a-repl)
+  * [Tutorials](#tutorials)
+  * [Contributing](#contributing)
+  * [License](#license)
+  * [Citation](#citation)
+  * [About Propeller](#about-propeller)
+  * [Contact](#contact)
+<!-- TOC -->
+
+### What can you do with Propeller?
+
+You can evolve a Push program to solve a problem. 
+You can also use the Push interpreter to evaluate Push programs in other projects, 
+for example in agent-based evolutionary simulations in which 
+agents are controlled by evolving Push programs.
+
+## Installation
+
+If you have installed [leiningen](https://leiningen.org), which is a tool
+for running Clojure programs, then you can run Propeller on a genetic
+programming problem that is defined within this project from the command
+line with the command `lein run -m <namespace>`, replacing `<namespace>`
+with the actual namespace that you will find at the top of the problem file.
+
+If you have installed [Clojure](https://clojure.org/guides/install_clojure#java), you can run Propeller on a genetic programming 
+problem with the command `clj -m <namespace>`, replacing `<namespace>` with 
+the actual namespace that you will find at the top of the problem file. 
+The examples below use leiningen, but you can replace `lein run -m` with `clj --main` to run the same problem.
+
+A specific example is provided later below.
+
+## How do I run Propeller on a problem?
+
+To run a problem in Propeller, you want to call the `-main` function in the problem file using leiningen. 
+The `-main` function will create a map of arguments from the input and run the main genetic programming loop.
+
+Below is the general format to run a problem through the command-line:
+
+```
+lein run -m [namespace of the problem file you want to test]
+```
+
+Additional command-line arguments may
+be provided to override the default key/value pairs specified in the
+problem file,
+
+```
+lein run -m [namespace of the problem file you want to test] [key and value] [key and value]...
+```
+
+The possible keys come from the table below:
+
+| Key                        | Description                                                                                                                                                                                                                                                                 |
+|----------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `:instructions`            | List of possible Push instructions used to create a plushy                                                                                                                                                                                                                  |
+| `:error-function`          | The error function used to evaluate individuals, specified in the given problem's namespace                                                                                                                                                                                 |
+| `:training-data`           | Map of inputs and desired outputs used to evaluate individuals of the form: {:input1 first-input :input2 second-input ... :output1 first-output ...}                                                                                                                        |
+| `:testing-data`            | Map of inputs and desired outputs not in the training-data to test generalizability of a program that fits the `training-data`. The map is of the form: {:input1 first-input :input2 second-input ... :output1 first-output ...}                                            |
+| `:max-generations`         | Maximum number of generations                                                                                                                                                                                                                                               |
+| `:population-size`         | Size of population in a generation                                                                                                                                                                                                                                          |
+| `:max-initial-plushy-size` | Maximum number of Push instructions in the initial plushy                                                                                                                                                                                                                   |
+| `:step-limit`              | The maximum number of steps that a Push program will be executed by `interpret-program`                                                                                                                                                                                     |
+| `:parent-selection`        | Function from `propeller.selection` that determines method of parent selection method. Propeller includes `:tournament-selection`, `:lexicase-selection`, and `:epsilon-lexicase-selection`.                                                                                |
+| `:tournament-size`         | If using a tournament selection method, the number of individuals in each tournaments used to determine parents                                                                                                                                                             |
+| `:umad-rate`               | Rate (decimal between 0 and 1) of uniform mutation by addition and deletion (UMAD) genetic operator                                                                                                                                                                         |
+| `:variation`               | Map with genetic operators as keys and probabilities as values. For example, {:umad 0.3 :crossover 0.7}. This would mean that when the system needs to generate a child, it will use UMAD 30% of the time and crossover 70% of the time. The probabilities should sum to 1. |
+| `:elitism`                 | When true, will cause the individual with the lowest error in the population to survive, without variation, into the next generation.                                                                                                                                       |
+
+When you run a problem, you will get a report each generation with the following information:
+
+```
+:generation            
+ :best-plushy 
+ :best-program          
+ :best-total-error      
+ :best-errors        
+ :best-behaviors        
+ :genotypic-diversity   
+ :behavioral-diversity  
+ :average-genome-length 
+ :average-total-error 
+
+```
+
+### An Example
+
+For example, you can run the simple-regression genetic programming problem with:
+
+```
+lein run -m propeller.problems.simple-regression
+```
+
+This will run simple-regression with the default set of arguments in the `simple-regression` problem file.
+
+```
+{:instructions             instructions
+:error-function           error-function
+:training-data            (:train train-and-test-data)
+:testing-data             (:test train-and-test-data)
+:max-generations          500
+:population-size          500
+:max-initial-plushy-size  100
+:step-limit               200
+:parent-selection         :lexicase
+:tournament-size          5
+:umad-rate                0.1
+:variation                {:umad 0.5 :crossover 0.5}
+:elitism                  false}
+```
+
+You can override the default key/value pairs with additional arguments. For example:
+
+```
+lein run -m propeller.problems.simple-regression :population-size 100
+```
+
+On Unix operating systems, including MacOS, you can use something
+like the following to send output both to the terminal
+and to a text file (called `outfile` in this example):
+
+```
+lein run -m propeller.problems.simple-regression | tee outfile
+```
+
+If you want to provide command line arguments that include
+characters that may be interpreted by your command line shell
+before they get to Clojure, then enclose those in double
+quotes, like in this example that provides a non-default
+value for the `:variation` argument, which is a clojure map
+containing curly brackets that may confuse your shell:
+
+```
+lein run -m propeller.problems.simple-regression :variation "{:umad 1.0}"
+```
+
+### Can you use a REPL?
+
+Yes!
+
+To run a genetic programming problem from a REPL, start
+your REPL for the project (e.g. with `lein repl` at the
+command line when in the project directory, or through your
+IDE) and then do something like the following (which in
+this case runs the simple-regression problem with
+`:population-size` 100):
+
+```
+(require 'propeller.problems.simple-regression)
+(in-ns 'propeller.problems.simple-regression)
+(-main :population-size 100 :variation {:umad 1.0})
+```
+
+If you want to run the problem with the default parameters,
+then you should call `-main` without arguments, as `(-main).
+
+## Tutorials
+
+- Adding genetic operators
+- Adding selection methods
+- Adding a new problem
+- Using Propeller for Experiments
+
+## Contributing
+
+You can report a bug on the [GitHub issues page](https://github.com/lspector/propeller/issues).
+
+The best way to contribute to Propeller is to fork the [main GitHub repository](https://github.com/lspector/propeller) and submit a pull request.
+
+## License
+
+**Eclipse Public License 2.0**
+
+This commercially-friendly copyleft license provides the ability to commercially license binaries; 
+a modern royalty-free patent license grant; and the ability for linked works to use other licenses, including commercial ones.
+
+## Citation
+
+We are in the process of creating a DOI, but in the meantime, 
+we ask that you cite the [link to the repository](https://github.com/lspector/propeller) if you use Propeller.
+
+## About Propeller
+
+Propeller was created by
+
+## Contact
+
+To discuss Propeller, Push, and PushGP, you can join the [Push-Language Discourse](https://discourse.pushlanguage.org/).

src/docs_src/Adding_Genetic_Operators.md

@@ -0,0 +1,62 @@
+# Adding Genetic Operators
+
+In addition to the already-included genetic operators, you can add your own!
+
+## Variation Genetic Operators
+
+1. Go to `propeller.variation.cljc`
+2. Define a genetic operator function
+3. In `propeller.variation/new-individual`, add the new genetic operator in the `new-individual` function under the `case` call
+
+``` clojure
+(defn new-individual
+  "Returns a new individual produced by selection and variation of
+  individuals in the population."
+  [pop argmap]
+  ...
+     (case op
+       ...
+       
+       :new-genetic-operator
+       (-> (:plushy (selection/select-parent pop argmap))
+           (new-genetic-operator ))
+       ...
+       :else
+       (throw #?(:clj  (Exception. (str "No match in new-individual for " op))
+                 :cljs (js/Error
+                         (str "No match in new-individual for " op))))))})
+```
+
+4. When running a problem, specify the genetic operator in `:variation`.
+For example:
+```
+lein run -m propeller.problems.simple-regression :variation "{:new-genetic-operator 1.0}"
+
+```
+
+## Selection Genetic Operators
+
+1. Go to `propeller.selection.cljc`
+2. Define a genetic operator function
+3. In `propeller.selection.cljc`, add the new genetic operator in the `select-parent` function under the `case` call.
+
+```clojure
+(defn select-parent
+  "Selects a parent from the population using the specified method."
+  [pop argmap]
+  (case (:parent-selection argmap)
+    ...
+    :new-genetic-operator (:new-genetic-operator )
+    ...
+    ))
+```
+
+4. When running a problem, specify the selection method in `:parent-selection`
+
+For example:
+
+```
+lein run -m propeller.problems.simple-regression :parent-selection :new-genetic-operator
+
+```
+

src/docs_src/Adding_Problem.md

@@ -0,0 +1,122 @@
+# Adding a Problem
+
+In general, a problem file has 3 components: `train-and-test-data`, `instructions`, `error-function`, and `-main`.
+
+1. To add a new problem, you need training and test data. For Problem Synthesis Benchmark Problems (PSB2),
+you can fetch datasets using `psb2.core/fetch-examples`.
+
+```clojure
+(defn fetch-examples
+  "Fetches and returns training and test data from a PSB2 problem.
+   Returns a map of the form {:train training-examples :test testing-examples}
+   where training-examples and testing-examples are lists of training and test
+   data. The elements of these lists are maps of the form:
+   {:input1 first-input :input2 second-input ... :output1 first-output ...}
+   The training examples will include all hard-coded edge cases included in the suite,
+   along with enough random cases to include `n-train` cases.
+   Note that this function loads large datasets and can be slow, 30-120 seconds.
+   Parameters:
+     `datasets-directory` - Location of the PSB2 datasets as downloaded from https://zenodo.org/record/4678739
+     `problem-name` - Name of the PSB2 problem, lowercase and seperated by dashes.
+         - Ex: indices-of-substring
+     `n-train` - Number of training cases to return
+     `n-test` - Number of test cases to return"
+  [datasets-directory problem-name n-train n-test]
+
+```
+2. Define the possible Push instructions to be used to create plushys. It should be a non-lazy list of 
+instructions from `push/instructions`
+3. Define an error function that will evaluate plushys and add `:behaviors parsed-outputs`, 
+   `:errors`, and `:total-error` to the individual
+4. Define the function `-main` with a map of default arguments.
+
+## Example of a Problem
+
+```clojure
+(ns propeller.problems.PSB2.solve-boolean
+  (:require [psb2.core :as psb2]
+            [propeller.genome :as genome]
+            [propeller.push.interpreter :as interpreter]
+            [propeller.utils :as utils]
+            [propeller.push.instructions :refer [get-stack-instructions]]
+            [propeller.push.state :as state]
+            [propeller.gp :as gp]
+            #?(:cljs [cljs.reader :refer [read-string]])))
+
+; ===========  PROBLEM DESCRIPTION  ================================
+; SOLVE BOOLEAN from PSB2
+; Given a string representing a Boolean
+; expression consisting of T, F, |, and &, evaluate it and return
+; the resulting Boolean.
+;
+; Source: https://arxiv.org/pdf/2106.06086.pdf
+; ==================================================================
+
+(def train-and-test-data (psb2/fetch-examples "data" "solve-boolean" 200 2000))
+
+(def instructions
+  (utils/not-lazy
+    (concat
+      ;;; stack-specific instructions
+      (get-stack-instructions #{:exec :integer :boolean :char :string :print})
+      ;;; input instructions
+      (list :in1)
+      ;;; close
+      (list 'close)
+      ;;; ERCs (constants)
+      (list true false \t \f \& \|))))
+
+(defn error-function
+  [argmap data individual]
+  (let [program (genome/plushy->push (:plushy individual) argmap)
+        inputs (map (fn [i] (get i :input1)) data)
+        correct-outputs (map (fn [i] (get i :output1)) data)
+        outputs (map (fn [input]
+                       (state/peek-stack
+                         (interpreter/interpret-program
+                           program
+                           (assoc state/empty-state :input {:in1 input})
+                           (:step-limit argmap))
+                         :boolean))
+                     inputs)
+        parsed-outputs (map (fn [output]
+                              (try (read-string output)
+                                   #?(:clj  (catch Exception e 1000.0)
+                                      :cljs (catch js/Error. e 1000.0))))
+                            outputs)
+        errors (map (fn [correct-output output]
+                      (if (= output :no-stack-item)
+                        10000
+                        (if (= correct-output output)
+                           0
+                           1)))
+                    correct-outputs
+                    parsed-outputs)]
+    (assoc individual
+      :behaviors parsed-outputs
+      :errors errors
+      :total-error #?(:clj  (apply +' errors)
+                      :cljs (apply + errors)))))
+
+(defn -main
+  "Runs propel-gp, giving it a map of arguments."
+  [& args]
+  (gp/gp
+    (merge
+      {:instructions            instructions
+       :error-function          error-function
+       :training-data           (:train train-and-test-data)
+       :testing-data            (:test train-and-test-data)
+       :max-generations         300
+       :population-size         1000
+       :max-initial-plushy-size 250
+       :step-limit              2000
+       :parent-selection        :lexicase
+       :tournament-size         5
+       :umad-rate               0.1
+       :variation               {:umad 1.0 :crossover 0.0}
+       :elitism                 false}
+      (apply hash-map (map #(if (string? %) (read-string %) %) args))))
+  (#?(:clj shutdown-agents)))
+
+```

src/docs_src/Adding_Selection_Method.md

@@ -0,0 +1,20 @@
+# Adding a Selection Method
+
+1. Define a selection method function in `propeller.selection` that selects an individual from the population
+2. Add the selection method in `propeller.selection/select-parent` under the `case` call:
+
+```clojure
+(defn select-parent
+  "Selects a parent from the population using the specified method."
+  [pop argmap]
+  (case (:parent-selection argmap)
+    :new-selection-method (new-selection-method )))
+```
+
+3. When runnning a problem, specify the selection method in `:parent-selection`. 
+For example:
+```
+lein run -m propeller.problems.simple-regression :parent-selection :new-selection-method"
+```
+
+

src/propeller/core.cljc

@@ -7,4 +7,4 @@
   ;; Exception for when no args were passed
   (println "To run a genetic programming problem, provide a the problem's")
   (println "namespace as specified in the Propeller README file at")
-  (println "https://github.com/lspector/propeller/blob/master/README.md"))
+  (println "https://github.com/lspector/propeller/blob/master/A_Guide_To_Propeller.md"))

src/propeller/genome.cljc

@@ -1,4 +1,7 @@
 (ns propeller.genome
+  "The genetic material in Propeller. A `plushy` is a list of Push instructions that represent a Push program.
+They hold the genetic material for an `individual`. In the initial population, we create random plushys."
+  {:doc/format :markdown}
   (:require [propeller.push.instructions :as instructions]
             [propeller.utils :as utils]))
 

src/propeller/problems/PSB2/basement.cljc

@@ -17,7 +17,7 @@
             [propeller.gp :as gp]
             #?(:cljs [cljs.reader :refer [read-string]])))
 
-(def train-and-test-data (psb2/fetch-examples "data" "basement" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "basement" 200 2000))
 
 (defn random-int
   "Random integer between -100 and 100 (from smallest)"

src/propeller/problems/PSB2/bouncing_balls.cljc

@@ -20,7 +20,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
             #?(:cljs [cljs.reader :refer [read-string]])))
 
 
-(def train-and-test-data (psb2/fetch-examples "data" "bouncing-balls" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "bouncing-balls" 200 2000))
 
 (defn map-vals-input
   "Returns all the input values of a map (specific helper method for bouncing-balls)"

src/propeller/problems/PSB2/bowling.cljc

@@ -19,7 +19,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
 
 
 
-(def train-and-test-data (psb2/fetch-examples "data" "bowling" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "bowling" 200 2000))
 
 (defn random-int "Returns random integer between -100 and 100" [] (- (rand-int 201) 100))
 

src/propeller/problems/PSB2/camel_case.cljc

@@ -19,7 +19,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
             #?(:cljs [cljs.reader :refer [read-string]])))
 
 
-(def train-and-test-data (psb2/fetch-examples "data" "camel-case" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "camel-case" 200 2000))
 
 ; Visible character ERC
 (defn random-char

src/propeller/problems/PSB2/dice_game.cljc

@@ -18,7 +18,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
             #?(:cljs [cljs.reader :refer [read-string]])))
 
 
-(def train-and-test-data (psb2/fetch-examples "data" "dice-game" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "dice-game" 200 2000))
 
 (defn map-vals-input
   "Returns all the input values of a map"

src/propeller/problems/PSB2/fizz_buzz.cljc

@@ -17,7 +17,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
             [propeller.gp :as gp]
             #?(:cljs [cljs.reader :refer [read-string]])))
 
-(def train-and-test-data (psb2/fetch-examples "data" "fizz-buzz" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "fizz-buzz" 200 2000))
 
 (def instructions
   "stack-specific instructions, input instructions, close, and constants"

src/propeller/problems/PSB2/fuel_cost.cljc

@@ -18,7 +18,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
             [propeller.gp :as gp]
             #?(:cljs [cljs.reader :refer [read-string]])))
 
-(def train-and-test-data (psb2/fetch-examples "data" "fuel-cost" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "fuel-cost" 200 2000))
 
 ; Random integer between -100 and 100 (from smallest)
 (defn random-int "Random integer between -100 and 100" [] (- (rand-int 201) 100))

src/propeller/problems/PSB2/gcd.cljc

@@ -16,7 +16,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
             [propeller.gp :as gp]
             #?(:cljs [cljs.reader :refer [read-string]])))
 
-(def train-and-test-data (psb2/fetch-examples "data" "gcd" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "gcd" 200 2000))
 
 (defn random-int "Random integer between -100 and 100" [] (- (rand-int 201) 100))
 

src/propeller/problems/PSB2/luhn.cljc

@@ -20,7 +20,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
             #?(:cljs [cljs.reader :refer [read-string]])))
 
 
-(def train-and-test-data (psb2/fetch-examples "data" "luhn" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "luhn" 200 2000))
 
 ; Random integer between -100 and 100 (from smallest)
 (defn random-int "Random integer between -100 and 100" [] (- (rand-int 201) 100))

src/propeller/problems/PSB2/middle_character.cljc

@@ -17,7 +17,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
             [propeller.gp :as gp]
             #?(:cljs [cljs.reader :refer [read-string]])))
 
-(def train-and-test-data (psb2/fetch-examples "data" "middle-character" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "middle-character" 200 2000))
 
 (defn random-int "Random integer between -100 and 100" [] (- (rand-int 201) 100))
 

src/propeller/problems/PSB2/paired_digits.cljc

@@ -16,7 +16,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
             [propeller.gp :as gp]
             #?(:cljs [cljs.reader :refer [read-string]])))
 
-(def train-and-test-data (psb2/fetch-examples "data" "paired-digits" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "paired-digits" 200 2000))
 
 (defn random-int "Random integer between -100 and 100" [] (- (rand-int 201) 100))
 

src/propeller/problems/PSB2/shopping_list.cljc

@@ -18,7 +18,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
             #?(:cljs [cljs.reader :refer [read-string]])))
 
 
-(def train-and-test-data (psb2/fetch-examples "data" "shopping-list" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "shopping-list" 200 2000))
 
 (defn random-float "Random float between -100 and 100" [] (- (rand 201) 100))
 

src/propeller/problems/PSB2/snow_day.cljc

@@ -21,7 +21,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
             [propeller.gp :as gp]
             #?(:cljs [cljs.reader :refer [read-string]])))
 
-(def train-and-test-data (psb2/fetch-examples "data" "snow-day" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "snow-day" 200 2000))
 
 (defn map-vals-input
   "Returns all the input values of a map"

src/propeller/problems/PSB2/solve_boolean.cljc

@@ -17,7 +17,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
             #?(:cljs [cljs.reader :refer [read-string]])))
 
 
-(def train-and-test-data (psb2/fetch-examples "data" "solve-boolean" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "solve-boolean" 200 2000))
 
 (def instructions
   "stack-specific instructions, input instructions, close, and constants"

src/propeller/problems/PSB2/spin_words.cljc

@@ -17,7 +17,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
             [propeller.gp :as gp]
             #?(:cljs [cljs.reader :refer [read-string]])))
 
-(def train-and-test-data (psb2/fetch-examples "data" "spin-words" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "spin-words" 200 2000))
 
 ; Visible character ERC
 (defn random-char

src/propeller/problems/PSB2/square_digits.cljc

@@ -18,7 +18,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
 
 
 
-(def train-and-test-data (psb2/fetch-examples "data" "square-digits" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "square-digits" 200 2000))
 
 (defn random-int "Random integer between -100 and 100" [] (- (rand-int 201) 100))
 

src/propeller/problems/PSB2/substitution_cipher.cljc

@@ -19,7 +19,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
             [propeller.gp :as gp]
             #?(:cljs [cljs.reader :refer [read-string]])))
 
-(def train-and-test-data (psb2/fetch-examples "data" "substitution-cipher" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "substitution-cipher" 200 2000))
 
 (defn map-vals-input
   "Returns all the input values of a map"

src/propeller/problems/PSB2/twitter.cljc

@@ -21,7 +21,7 @@ Source: https://arxiv.org/pdf/2106.06086.pdf"
 
 
 
-(def train-and-test-data (psb2/fetch-examples "data" "twitter" 200 2000))
+(def train-and-test-data "Data taken from https://zenodo.org/record/5084812" (psb2/fetch-examples "data" "twitter" 200 2000))
 
 (defn random-int "Random integer between -100 and 100" [] (- (rand-int 201) 100))
 

src/propeller/problems/simple_regression.cljc

@@ -16,6 +16,9 @@ Given inputs and outputs, find the target function."
   (+ (* x x x) x 3))
 
 (def train-and-test-data
+  "Training data: Inputs and outputs with -10 <= x < 11
+
+  Test data: Inputs and outputs of -20 <= x < -10 and 11 <= x < 21"
   (let [train-inputs (range -10 11)
         test-inputs (concat (range -20 -10) (range 11 21))]
     {:train (map (fn [x] {:input1 (vector x) :output1 (vector (target-function x))}) train-inputs)

src/propeller/problems/string_classification.cljc

@@ -46,6 +46,9 @@ Given a string, return true if it contains A, C, G, and T. Else return false."
         "T"))
 
 (def train-and-test-data
+  "Training data: \"GCG\" \"GACAG\" \"AGAAG\" \"CCCA\" \"GATTACA\" \"TAGG\" \"GACT\" with associated boolean values based on problem definition.
+
+  Test data: \"GCGT\" \"GACTTAG\" \"AGTAAG\" \"TCCTCA\" \"GAACA\" \"AGG\" \"GAC\" with associated boolean values based on problem definition."
   (let [train-inputs ["GCG" "GACAG" "AGAAG" "CCCA" "GATTACA" "TAGG" "GACT"]
         test-inputs ["GCGT" "GACTTAG" "AGTAAG" "TCCTCA" "GAACA" "AGG" "GAC"]
         train-outputs [false false false false true true true]

src/propeller/push/instructions.cljc

@@ -5,12 +5,16 @@
             #?(:cljs [goog.string :as gstring])
             #?(:cljs [goog.string.format])))
 
-;; PushGP instructions are represented as keywords, and stored in an atom. They
-;; can be either constant literals or functions that take and return a Push state
-(def instruction-table (atom (hash-map)))
+
+(def instruction-table
+  "PushGP instructions are represented as keywords, and stored in an atom. They
+can be either constant literals or functions that take and return a Push state"
+  (atom (hash-map)))
 
 ;; Number of blocks opened by instructions (default = 0)
-(def opens {:exec_dup 1
+(def opens
+  "Number of blocks opened by instructions. The default is 0."
+  {:exec_dup 1
             :exec_dup_times 1
             :exec_dup_items 0 ; explicitly set to 0 to make it clear that this is intended
             :exec_eq 0 ; explicitly set to 0 to make it clear that this is intended

src/propeller/push/instructions/character.cljc

@@ -1,4 +1,5 @@
 (ns propeller.push.instructions.character
+  "Push instructions for CHARs."
   (:require [propeller.push.state :as state]
             [propeller.tools.character :as char]
             [propeller.push.instructions :refer [def-instruction

src/propeller/push/instructions/code.cljc

@@ -1,4 +1,5 @@
 (ns propeller.push.instructions.code
+  "Push instructions for code."
   (:require [propeller.utils :as utils]
             [propeller.push.state :as state]
             [propeller.push.instructions :refer [def-instruction

src/propeller/push/instructions/input_output.cljc

@@ -1,4 +1,5 @@
 (ns propeller.push.instructions.input-output
+  "Push instructions for input and output."
   (:require [propeller.push.state :as state]
             [propeller.push.instructions :refer [def-instruction
                                                  generate-instructions]]))

src/propeller/push/instructions/vector.cljc

@@ -1,4 +1,5 @@
 (ns propeller.push.instructions.vector
+  "Vector instructions for all vector element subtypes: BOOLEAN, FLOAT, INTEGER, and STRING."
   (:require [clojure.string]
             [propeller.utils :as utils]
             [propeller.push.state :as state]

src/propeller/push/limits.cljc

@@ -1,4 +1,6 @@
 (ns propeller.push.limits
+  "Values used by the Push instructions to keep the stack sizes within reasonable limits
+  and values used by the Push instructions to keep computed values within reasonable size limits."
   (:require [propeller.utils :as u]))
 
 ;; =============================================================================
@@ -9,7 +11,11 @@
 ;; Limits the number of items that can be duplicated onto a stack at once.
 ;; We might want to extend this to limit all the different that things may be
 ;; placed on a stack.
-(def max-stack-items 100)
+(def max-stack-items
+  "Limits the number of items that can be duplicated onto a stack at once.
+We might want to extend this to limit all the different that things may
+be placed on a stack."
+  100)
 
 ;; =============================================================================
 ;; Values used by the Push instructions to keep computed values within
@@ -17,27 +23,32 @@
 ;; =============================================================================
 
 ;; Used as the maximum magnitude of any integer/float
-(def max-number-magnitude 1.0E6)
+(def max-number-magnitude
+  "Used as the maximum magnitude of any integer/float."
+  1.0E6)
 
 ;; Used as the minimum magnitude of any float
-(def min-number-magnitude 1.0E-6)
+(def min-number-magnitude
+  "Used as the minimum magnitude of any float."
+  1.0E-6)
 
 ;; Used to ensure that strings don't get too large
-(def max-string-length 1000)
+(def max-string-length "Used to ensure that strings don't get too large." 1000)
 
 ;; Used to ensure that vectors don't get too large
-(def max-vector-length 1000)
+(def max-vector-length "Used to ensure that vectors don't get too large." 1000)
 
 ;; Used to ensure that total
 ;; Set as dynamic for testing purposes.
-(def ^:dynamic max-code-points 100)
+(def ^:dynamic max-code-points "Used to ensure that total code points don't get too large. Set as dynamic for testing purposes." 100)
 
 ;; Used to ensure that the depth of nesting for Push expressions doesn't get too deep.
 ;; Set as dynamic for testing purposes.
-(def ^:dynamic max-code-depth 200)
+(def ^:dynamic max-code-depth "Used to ensure that the depth of nesting for Push expressions doesn't get too deep. Set as dynamic for testing purposes." 200)
 
 ;; Returns a version of the number n that is within reasonable size bounds
 (defn limit-number
+  "Returns a version of the number n that is within reasonable size bounds."
   [n]
   (if (int? n)
     (cond
@@ -57,14 +68,17 @@
       :else n)))
 
 (defn limit-string
+  "Limits string length to max-string-length."
   [s]
   (apply str (take max-string-length s)))
 
 (defn limit-vector
+  "Limits vector length to max-vector-length."
   [v]
   (vec (take max-vector-length v)))
 
 (defn limit-code
+  "Limits code to max-code-points and max-code-depth."
   [code]
   (if (or (> (u/count-points code) max-code-points)
           (> (u/depth code) max-code-depth))

src/propeller/selection.cljc

@@ -1,4 +1,7 @@
 (ns propeller.selection
+  "Propeller includes many kinds of genetic operators to select parents within the population such as tournament selection,
+  lexicase selection, and epsilon lexicase selection."
+  {:doc/format :markdown}
   (:require [propeller.tools.math :as math-tools]))
 
 (defn tournament-selection

src/propeller/session.cljc

@@ -1,11 +1,15 @@
 ; The "session" namespace is for trying things out interactively.
 ; For example, you can use it to test a new Push instruction by running a program that uses it and seeing the result.
-; You might just want to do this interactively in the REPL, but the session file makes it a little easier since it alerady
+; You might just want to do this interactively in the REPL, but the session file makes it a little easier since it already
 ; requires most of the namespaces you'll want to refer to.
 ; The commented-out stuff is a reminder of how to do some basic things.
 
 
 (ns propeller.session
+  "The \"session\" namespace is for trying things out interactively.
+  For example, you can use it to test a new Push instruction by running a program that uses it and seeing the result.
+  You might just want to do this interactively in the REPL, but the session file makes it a little easier since it already
+  requires most of the namespaces you'll want to refer to."
   (:require [propeller.genome :as genome]
             [propeller.gp :as gp]
             [propeller.selection :as selection]

src/propeller/simplification.cljc

@@ -1,4 +1,28 @@
 (ns propeller.simplification
+  "To use Propeller's auto-simplification system, simply include the following four command line arguments when running a problem:
+
+```clojure
+:simplification? true
+```
+Toggle auto-simplification
+```clojure
+:simplification-k 4
+```
+This is the upper bound for elements deleted from the plushy every step. Every step, a number in [1, k] of elements is deleted from the plushy representation of the solution.
+```clojure
+:simplification-steps 1000
+```
+Number of simplification steps to perform
+```clojure
+:simplification-verbose? true
+```
+Whether or not to output simplification info into the output of the evolutionary run.
+The output with verbose adds the following lines to the output:
+```clojure
+{:start-plushy-length 42, :k 4}
+{:final-plushy-length 13, :final-plushy (:in1 :in1 :integer_quot :in1 :in1 :exec_dup :in1 :integer_mult close :exec_dup :integer_add 1 :integer_add)}
+```"
+  {:doc/format :markdown}
   (:require [propeller.genome :as genome]
              [propeller.push.interpreter :as interpreter]
             [propeller.push.state :as state]

src/propeller/utils.cljc

@@ -1,5 +1,5 @@
 (ns propeller.utils
-  "Useful functions"
+  "Useful functions."
   (:require [clojure.zip :as zip]))
 
 (defn first-non-nil