Decanter uses the CL-PPCRE library for regular expressions. Review its documentation for details on the regular expression format and how to write them.

You can include regular expressions in a URL pattern by marking it with the :regex keyword, either in the defurls macro or when using the add-mapping method:

(defurls *urls*
    '("/(.*)" :regex placeholder))

(add-mapping *app* "/hello/(.*)" hello-name :regex t)

Regular expressions are used when matching a requested URL against a URL pattern. They can capture substrings, which are provided to handler functions in the pattern-matches parameter.

For example:

(defhandler placeholder
  (:get ()
        (format t "~a" pattern-matches)
        "Placeholder"))

Prints a plist of the form:

(:type :regex
 :match ...
 :substrings #(...))

Syntax Sugar

Decanter includes a read macro to make specifying regular expression URL patterns a bit cleaner. Since read macros defined by different systems can collide with each other, it is disabled by default.

The read macro works within defurls and must be enabled before calling defurls:

(enable-regex-sugar)

(defurls *urls*
    '(#?"/(.*)" hello))

This is equivalent to:

(defurls *urls*
    '("/(.*)" :regex hello))

You can add variable rules to a URL pattern by marking them with (variable-name). Your handler function then takes the variable-name as an argument to its request method forms. Optionally, you can also specify a type for the variable rule like (variable-name :float), which will limit what the rule matches against.

Note that variable rules can perform type checking, but not type conversions. Their values are passed as strings to handler functions, which may perform the conversions later.

(url-regex-sugar)

(defurls *urls*
    '(#?"/(.*)" placeholder
      "/hello/(name)/" hello
      "/vector/(x :float)/(y :float)/(z :float)/" vec))

(defapp *app* *urls*)

(defhandler placeholder
  (:get () ""))

(defhandler hello
  (:get (name)
        (format nil "Hello, ~a!" name)))

(defhandler vec
  (:get (x y z)
        (format nil "<~a, ~a, ~a>" x y z)))

Valid types for variable rules are as follows:

The following URL patterns differ in their use of a trailing slash:

(defurls *urls*
    '("/projects/" projects
      "/about" about))

The URL for the projects handler function has a trailing slash. If you access this URL without a trailing slash, Decanter raises an error or replies with a 404 "Not Found" error, depending on how the application is configured.

The URL for the about handler function does not have a trailing slash. Accessing this URL with a trailing slash causes Decanter to raise an error or reply with a 404 "Not Found" error, depending on how the application is configured.

We recommend sticking to a convention with your URLs: either always use trailing slashes or never use them. Conventions help keep websites clean and predictable.

To build a URL to a specific handler, use the handler-url macro. It accepts an application and the name of a handler as its first arguments, and any number of subsequent keyword arguments, each corresponding to a variable rule in a URL pattern. Unknown keyword arguments are appended to the URL as query parameters.

When the pattern is instead a :regex pattern, the pattern string is returned with no modifications.

Why would you want to build URLs using the URL reversing macro handler-url, instead of hard-coding them into your code?

1. Reversing is often more descriptive than hard-coding the URLs.
2. You can change your URLs in one go instead of needing to remember to manually change hard-coded URLs.
3. URL building handles escaping of special characters transparently.
4. The generated paths are always absolute, avoiding unexpected behavior of relative paths in browsers.

For example:

(require :decanter)
(use-package :decanter)

(defurls *urls*
    '("/" index
      "/login/" login
      "/user/(username)/" user-profile
      "/print-handlers/" print-handlers))

(defapp *app* *urls*)

(defhandler index
  (:get () "index"))

(defhandler login
  (:get () "login"))

(defhandler user-profile
  (:get (username) (format nil "~a's profile" username)))

(defhandler print-handlers
  (:get ()
        (format t "~s~%" (handler-url index))
        (format t "~s~%" (handler-url login))
        (format t "~s~%" (handler-url login :next "/"))
        (format t "~s~%" (handler-url user-profile :username "John Doe"))
        "print-handlers"))

(run *app*)

(format t "~s~%" (handler-url *app* index))
(format t "~s~%" (handler-url *app* login))
(format t "~s~%" (handler-url *app* login :next "/"))
(format t "~s~%" (handler-url *app* user-profile :username "John Doe"))

"/"
"/login/"
"/login/?next=%2F"
"/user/John%20Doe/"

When using handler-url inside a defhandler body, you should omit the application parameter.

Decanter can accept HTTP requests using any HTTP request method. Specifying how to handle different HTTP request methods is easy using the defhandler macro:

(defhandler hello
  (:get () "Hello, get!")
  (:head () "Hello, head!")
  (:post () "Hello, post!")
  (:put () "Hello, put!")
  (:delete () "Hello, delete!")
  (:connect () "Hello, connect!")
  (:options () "Hello, options!")
  (:trace () "Hello, trace!")
  (:patch () "Hello, patch!"))

Recall that a handler function is associated with a URL pattern. It is therefore straightforward to handle multiple different HTTP request methods on the same URL pattern.

A Lisp form within a defhandler declaration that is associated with an HTTP request method is called a subhandler:

(:get () "Hello, get!")

A subhandler consists of a keyword corresponding to an HTTP request method, a list of HTTP request parameters and URL variable rule values, and a body of Lisp forms. The body should always evaluate to either a string or an HTTP response object.

Subhandlers can capture HTTP request parameters and URL variable rule values as variables. In the event there is a name collision between these two categories, an error is either raised or a 400 "Bad Request" is returned to the client.

URL variable rule values are always captured. Only query string parameters are captured from GET, HEAD, DELETE, OPTIONS, and TRACE requests. Parameters specified in the request body are captured from POST, PUT, and PATCH requests (e.g. from a web form); if a query string is also present, it is captured with the name request-query-parameters. No parameters are captured from CONNECT requests.

(url-regex-sugar)

(defurls *urls*
    '(#?"/(.*)" hello
      "/add(.*)" :regex add
      "/vector/(x :float)/(y :float)/(z :float)/" vec))

(defapp *app* *urls*)

(defhandler hello
  (:get ((name :string))
        (when (not name)
          (setf name "World"))
        (format nil "Hello, ~a!" name)))

(defhandler add
  (:get ((n :integer :required) (m :integer :required))
        (format nil "~a" (+ n m))))

(defhandler vec
  (:get ((x :float :required)
         (y :float :required)
         (z :float :required))
        (format nil "<~a, ~a, ~a>" x y z)))

So that:

Subhandler variables are strings or nil by default.

(defhandler hello
  (:get (name)
        (when (not name)
          (setf name "World"))
        (format nil "Hello, ~a!" name)))

Adding a type specifier causes a subhandler variable, when it is not nil, to be parsed into that type.

(defhandler hello
  (:get ((name :string))
        (when (not name)
          (setf name "World"))
        (format nil "Hello, ~a!" name)))

(defhandler add
  (:get ((n :integer) (m :integer))
        (format nil "~a" (+ n m))))

You can mandate that certain subhandler variables are required. Values of nil are rejected.

(defhandler add
  (:get ((n :integer :required) (m :integer :required))
        (format nil "~a" (+ n m))))

Lastly, you may associate a predicate with a subhandler variable. When a predicate is specified along with a type, the predicate expects a variable of that type as its parameter. Otherwise, it expects a string parameter.

The predicate is called on the variable. If the predicate returns a non-nil value, the subhandler body is evaluated and an HTTP response is returned as normal. If the predicate returns nil, the body is not evaluated. An error is either raised or a 400 "Bad Request" response is returned to the client, depending on how the Decanter application is configured.

The use of predicates can help with server-side input validation.

(defun string-or-nil-p (var)
  (or (null var)
      (stringp var)))

(defhandler hello
  (:get ((name #'string-or-nil-p))
        (when (not name)
          (setf name "World"))
        (format nil "Hello, ~a!" name)))

(defun in-range-p (var)
  (and (>= var 0) (< var 10)))

(defhandler add
  (:get ((n :integer #'in-range-p :required)
         (m :integer #'in-range-p :required))
        (format nil "~a" (+ n m))))

Order matters. The following are valid subhandler variable lists:

()
(var)
((var :required))
((var :integer))
((var :integer :required))
((var :integer #'in-range-p))
((var :integer #'in-range-p :required))

Of course, more than one variable can be specified.

Request objects are instances of the CLOS class request, which is defined within Decanter.

A request object named request is always passed to and available within handler functions. When using defhandler to define handler functions, this is done automatically.

A request object has the following fields and accessors:

method

A keyword representing the HTTP request method.

Valid request methods are: :get, :head, :post, :put, :delete, :connect, :options, :trace, and :patch.

(request-method request)

Returns the HTTP request method from the object request as a keyword: :get, :post, etc.

(setf (request-method request) value)

Sets the HTTP request method on the object request to a keyword value.

target

A string containing the HTTP request target. When applicable, this may include a path, query, or fragment.

For example:

• /
• /index.html
• /hello?name=person
• /about#section
• /person/?id=100#name

(target request)

Returns the HTTP request target from the object request as a string.

(setf (target request) value)

Sets the HTTP request target on the object request to a string value.

headers

A plist containing the HTTP request headers.

For example: (:content-type "text/plain" :content-length "100")

(headers request)

Returns the HTTP request headers from the object request as a plist.

(setf (headers request) value)

Sets the HTTP request headers on the object request to a plist value.

body

An octet vector containing the HTTP request body.

The vector has the type (simple-array (unsigned-byte 8) (length)), where length is the length of the vector.

(body request)

Returns the HTTP request body from the object request as an octet vector.

(body-string request)

Returns the HTTP request body from the object request as a string.

(setf (body request) value)

Sets the HTTP request body on the object request to an octet vector value.

cookies

An alist of string keys and string values containing the cookies sent by the client.

(cookies request)

Returns the cookies from the object request as an alist.

(setf (cookies request) value)

Sets the cookies on the object request to an alist value.

(get-cookie request key)

Gets a cookie value from the object request with the key.

clack-env

A plist containing the Clack environment.

Clack operates at a lower level of abstraction than Decanter, but there may be situations where access to the Clack environment is useful to a user.

(clack-env request)

Returns the Clack environment from the request body as a plist.

When a URL pattern matches against a requested URL, it collects some data. That data is provided to the handler function in the parameter pattern-matches. This parameter is a plist with a structure that depends on the type of URL pattern.

With a regular expression URL pattern:

(defurls *urls*
    '("/(.*)" :regex handler))

(defhandler handler
  (:get () (format nil "~s" pattern-matches)))

pattern-matches looks like the following, with :match and :substrings captured by the CL-PPCRE library function scan-to-strings:

(:type :regex
 :match ...
 :substrings #(...))

With a URL pattern containing variable rules:

(defurls *urls*
    '("/vector/(x :float)/(y :float)/(z :float)/" vec))

(defhandler vec
  (:get () (format nil "~s" pattern-matches)))

pattern-matches looks like the following:

(:type :var-rules
 :total-chars-matched ...
 :variables ((:name ...
              :type ...
              :value ...)
             ...))

A variable will not always have a :type:, but it will always have a :name and a :value.

You can handle file uploads with Decanter easily. When you're using an HTML form, make sure to set the enctype="multipart/form-data" attribute on that form, otherwise the browser will not transmit your files.

An uploaded file is captured as a variable in a handler function. A file variable is a list containing an input stream, a filename, and a file type; the stream holds the file's contents, while the filename and file type are strings.

Note that a filename is user-specified input, and you should neither trust its value nor make use of it without validation and cleaning.

Decanter provides two utility functions to make working with file streams easy:

• file-to-string
• file-to-vector

Each function reads the stream in a file variable, returning the file's contents as a string or an octet vector, respectively.

Here is an example showing how it all works. In this example, we echo a text file back to its uploader:

(require :decanter)
(use-package :decanter)

(defurls *urls*
    '("/" file))

(defapp *app* *urls*)

(defhandler file
  (:post (file)
         (destructuring-bind (file-stream file-name file-type)
             file
           (declare (ignore file-name file-type))
           (format nil "~a" (file-to-string file-stream)))))

(run *app*)

We'll send a text file using an HTTP client, Dexador. The text file is named file.txt and consists of one line: file contents.

(require :dexador)

(dex:post "http://localhost:5000"
          :content '((:file . #P"file.txt")))

Our file variable looks like this:

'(#<FLEXI-STREAMS::VECTOR-INPUT-STREAM {102F7377F3}>
  "file.txt"
  "text/plain")

The handler function returns this:

"file contents"

Instead, we can use file-to-vector:

(defhandler file
  (:post (file)
         (destructuring-bind (file-stream file-name file-type)
             file
           (declare (ignore file-name file-type))
           (format nil "~a" (file-to-vector file-stream)))))

In which case the handler function returns:

"#(102 105 108 101 32 99 111 110 116 101 110 116 115)"

To access cookies you can use the get-cookies method on a request object. To set cookies you can use the set-cookie method on a response object. The cookies field of a request object is an alist with all of the cookies the client transmits.

If you want to use sessions, do not use the cookies directly, but instead use the Sessions in Decanter that add some security on top of cookies.

Reading cookies:

(defhandler index
  (:get ()
        (format nil "Hello, ~a!"
                (get-cookie request "username"))))

Storing cookies:

(defhandler index
  (:get ()
        (let ((response (response-200)))
          (set-cookie response
                      "username"
                      :value "the username")
          response)))

Note that cookies are set on response objects, which can be returned from handler functions just like strings.

set-cookie takes the following parameters:

(set-cookie (response key
                      &key
                      (value "")
                      expires
                      (path "/")
                      domain
                      secure
                      httponly
                      samesite))

• response (response) - the response object where we are setting the cookie.
• key (string) - the key (name) of the cookie to be set.
• value (string) - the value of the cookie.
• expires (universal time or nil) - when the cookie expires.
• path (string) - limits the cookie to a given path, per default it will span the whole domain.
• domain (string or nil) - if you want to set a cross-domain cookie. For example, "example.com" will set a cookie that is readable by the domain www.example.com, foo.example.com, etc. Otherwise, a cookie will only be readable by the domain that set it.
• secure (t or nil) - if t, the cookie will only be available via HTTPS.
• httponly (t or nil) - disallow JavaScript access to the cookie.
• samesite (:lax, :strict, or nil) - limit the scope of the cookie to only be attached to requests that are "same-site".

Redirecting a request is as simple as returning a response object from the handler function with a 303 status code and an appropriate location header. Decanter provides constructors that make doing so easy:

(response-303 location)

Returns a new response object with a 303 status code and the location header with the specified location, which is a string containing a URL.

(response-redirect location)

An alias for response-303. It behaves identically.

Returning an HTTP error is straightforward and involves returning a response object from the handler function with an appropriate status code. Decanter provides constructors that make doing so easy:

(response-400 &optional headers body)

Returns a new response object with a 400 status code, the content-type header set to "text/plain" and the body set to "Bad Request".

The user may optionally specify different headers and a different body.

(response-404 &optional headers body)

Returns a new response object with a 404 status code, the content-type header set to "text/plain" and the body set to "Not Found".

The user may optionally specify different headers and a different body.

(response-500 &optional headers body)

Returns a new response object with a 500 status code, the content-type header set to "text/plain" and the body set to "Internal Server Error".

The user may optionally specify different headers and a different body.

(response-bad-request &optional headers body)

An alias for response-400. It behaves identically.

(response-not-found &optional headers body)

An alias for response-404. It behaves identically.

(response-error &optional headers body)

An alias for response-500. It behaves identically.

JSON is a common response format when writing an API. Decanter allows you to write this type of API easily. If you return a list from a handler function, it will be converted to a JSON response.

(require :decanter)
(use-package :decanter)

(defurls *urls*
    '("/json/" json))

(defapp *app* *urls*)

(defhandler json
  (:get ()
        '(:array

          (:object :name "user 1" :profile-image "image-1.jpg")

          (:object ("name" . "user 2") ("profile-image" . "image-2.jpg"))

          (:array 1 2 3)

          #(1 2 3)

          t

          nil

          "hello"

          :true

          :false

          :null)))

(run *app*)

[
    {
        "name": "user 1",
        "profileImage": "image-1.jpg"
    },
    {
        "name": "user 2",
        "profile-image": "image-2.jpg"
    },
    [
        1,
        2,
        3
    ],
    [
        1,
        2,
        3
    ],
    true,
    null,
    "hello",
    "true",
    "false",
    "null"
]

(defhandler json
  (:get ()
        '(:object
          :name "user 1"
          :profile-image "image-1.jpg")))

{
    "name": "user 1",
    "profileImage": "image-1.jpg"
}

Decanter will convert any supported data type to JSON, which means that any data in the list must be convertible by cl-json.

Note that a list can be converted to either an :array or an :object in JSON, depending on what keyword you specify as the first element in the list. When working with lists, you must always specify a keyword.

Both association lists and property lists may be converted to json objects.

The following functions help you work with sessions:

(session-id request)

Returns the session id, which is a string.

(session-options request)

Returns a plist of session options, e.g.

(:ID "..."
 :NEW-SESSION T
 :CHANGE-ID NIL
 :EXPIRE NIL)

(session-table request)

Returns the hash table associated with the session, which can be used to store data across requests.

(session request key)

Returns a value from the session hash table, keyed by the string key.

(set-session request key value)

Sets a value in the session hash table, keyed by the string key.

(rem-session request key)

Removes a value from the session hash table, keyed by the string key.

(clear-session request)

Deletes all keys and values from the session hash table.

When calling these functions within the body of a defhandler, omit the request parameter.

For example:

(defhandler session-id
  (:get () (session-id)))

In either case, when you call the run function, you must enable standalone functionality:

(run *app* :standalone t)

You will almost certainly want to enable production mode, which alters error handling behavior to send HTTP responses with error codes instead of raising Lisp errors that can pause or terminate execution of the application:

(defapp *app *urls* :production t)

(run *app* :standalone t)

Or:

(setf (production *app*) t)

(run *app* :standalone t)

You may also wish to disable the printing of debug messages:

(defapp *app *urls*
  :production t
  :debug nil)

(run *app* :standalone t)

Or:

(setf (production *app*) t
      (application-debug *app*) nil)

(run *app* :standalone t)

Note that deployment of a shell script means that SBCL will need to be installed on the target system, along with all of your application's dependencies. While the Common Lisp ecosystem is generally fairly stable, this may not be ideal.

Create a shell script:

#!/usr/bin/env -S sbcl --script
(load "~/.sbclrc")

(require :decanter)
(use-package :decanter)

(defurls *urls*
    '("/" hello))

(defapp *app* *urls*
  :production t
  :debug nil)

(defhandler hello
  (:get () "Hello, world!"))

(run *app* :standalone t)

Creating a standalone binary can load all of an application's dependencies into a Lisp image, which is then immediately saved in an executable format. Therefore, packaging an application in this way requires no dependencies to be installed on the target system, other than those external to Lisp itself, e.g. a database system.

Create a standalone binary:

(require :decanter)
(use-package :decanter)

(defurls *urls*
    '("/" hello))

(defapp *app* *urls*
  :production t
  :debug nil)

(defhandler hello
  (:get () "Hello, world!"))

(defun executable-entry-point ()
  (run *app* :standalone t))

Save the above in a file, e.g. hello.lisp. Then tell SBCL to load it and build a binary.

$ sbcl \
  --noinform --non-interactive \
  --eval "(load \"hello.lisp\")" \
  --eval "(sb-ext:save-lisp-and-die #p\"hello\" :toplevel #'executable-entry-point :executable t)"

You can run the binary like any other:

$ chmod +x hello
$ ./hello

A shell script or binary may then be automatically started by the operating system, for example by a systemd service, or otherwise deployed as you prefer.

The various Lisp implementations offer different types of functionality in this vein. Check your implementation's documentation to get a better sense of what features it makes available.

Consider an application with the following structure:

/application
    /src
        main.lisp
        models.lisp
        views.lisp
        templates.lisp
    /tests
        tests.lisp
    /static
        style.css
    /templates
        page.lisp
        index.lisp
        login.lisp
        logout.lisp
        ...

Each file in the application/src directory defines a package:

application/src/main.lisp      => :application
application/src/models.lisp    => :application.models
application/src/views.lisp     => :application.views
application/src/templates.lisp => :application.templates

models.lisp contains functions that interact with a database. Functions exported by this package are used in views.lisp, where defhandler defines multiple handler functions. views.lisp also makes use of templates, which are just functions wrapping Spinneret markup, that return HTML strings. Templates are exported in templates.lisp.

main.lisp contains the application entry point, URL definitions in defurl, and defapp. Since URLs map to handler functions, it depends on views.lisp.

Individual templates reside inside files within the application/src/templates/ directory, but they are still part of the templates package, with (in-package :application.templates) as their first form. templates.lisp just exports each of these templates.

This pattern of splitting templates into their own files and exporting them using a thin templates.lisp file can make it easier to keep an application organized. The same is true for splitting models.lisp and views.lisp into their own directories and files, though for this example we haven't done this.

We'll make an ASD file for this system, which will make all four of its constituent packages available:

(defsystem "application"
  :description "An example Decanter application."
  :version "0.1.0"
  :author "Application Authors"
  :license "BSD-2"
  :depends-on ("decanter"
               "decanter.ext.db")
  :components (;; Templates

               (:file "application.templates"
                :pathname "src/templates")

               (:file "application.templates.page"
                :pathname "templates/page"
                :depends-on ("application.templates"))

               (:file "application.templates.index"
                :pathname "templates/index"
                :depends-on ("application.templates"))

               (:file "application.templates.login"
                :pathname "templates/login"
                :depends-on ("application.templates"))

               (:file "application.templates.logout"
                :pathname "templates/logout"
                :depends-on ("application.templates"))

               ;; Models

               (:file "application.models"
                :pathname "src/models")

               ;; Views

               (:file "application.views"
                :pathname "src/views"
                :depends-on ("application.templates"
                             "application.models"))

               ;; Application

               (:file "application"
                :pathname "src/main"
                :depends-on ("application.views"))))

The application depends on decanter and, since it makes use of a database, we have also added decanter.ext.db as a dependency.

After saving this file as application.asd, our application structure looks like this:

/application
    application.asd
    /src
        main.lisp
        models.lisp
        views.lisp
        templates.lisp
    /tests
        tests.lisp
    /static
        style.css
    /templates
        page.lisp
        index.lisp
        login.lisp
        logout.lisp
        ...

Your Lisp implementation needs to know where to find the system. You can achieve this by creating a symbolic link in one of two places: the Lisp system source path, or Quicklisp's local projects directory.

For example, on Linux:

$ ln -s ~/application ~/.local/share/common-lisp/source/

$ ln -s ~/application ~/quicklisp/local-projects/

Remember that static files are loaded from the file system by Decanter's internals, so we don't need to include them as part of the system definition. We do, however, need to make sure to set the right application directory and static directory in defapp, e.g.

(defapp *app* *urls*
  :application-directory #P"~/application/"
  :static-directory #P"~/application/static/")

Alternatively, you can specify paths relative to your ASDF system's root, which is where application.asd resides:

(defparameter *system-dir*
  (asdf:system-source-directory :application))

(defparameter *static-dir*
  (asdf:system-relative-pathname :application "static/"))

(defapp *app* *urls*
  :application-directory *system-dir*
  :static-directory *static-dir*)

Or relative to the current working directory using UIOP, which is part of ASDF:

(defparameter *system-dir* (uiop/os:getcwd))
(defparameter *static-dir* (merge-pathnames #P"static/" *system-dir*))

(defapp *app* *urls*
  :application-directory *system-dir*
  :static-directory *static-dir*)

In that case, remember to start your Lisp implementation in your application's directory, or set the current working directory once the Lisp is loaded, like this:

(uiop:chdir "~/application/")

With Quicklisp:

(ql:quickload :application)

With ASDF:

(asdf:load-system :application)

Sometimes you might want to force a complete rebuild of the system. You can do this with the :force option:

(asdf:load-system :application :force t)

Then, you can require the application's packages:

(require :application)

(require :application.models)

(require :application.views)

(require :application.templates)

Let's set up the ASD file to run some tests:

(defsystem "application"
  :description "An example Decanter application."
  :version "0.1.0"
  :author "Application Authors"
  :license "BSD-2"
  :depends-on ("decanter"
               "decanter.ext.db")
  :components (;; Templates

               (:file "application.templates"
                :pathname "src/templates")

               (:file "application.templates.page"
                :pathname "templates/page"
                :depends-on ("application.templates"))

               (:file "application.templates.index"
                :pathname "templates/index"
                :depends-on ("application.templates"))

               (:file "application.templates.login"
                :pathname "templates/login"
                :depends-on ("application.templates"))

               (:file "application.templates.logout"
                :pathname "templates/logout"
                :depends-on ("application.templates"))

               ;; Models

               (:file "application.models"
                :pathname "src/models")

               ;; Views

               (:file "application.views"
                :pathname "src/views"
                :depends-on ("application.templates"
                             "application.models"))

               ;; Application

               (:file "application"
                :pathname "src/main"
                :depends-on ("application.views")))
  :in-order-to ((test-op (test-op "application.tests"))))

(defsystem "application.tests"
  :description "Tests for an example Decanter application."
  :version "0.1.0"
  :author "Application Authors"
  :license "BSD-2"
  :depends-on ("application")
  :components ((:file "tests" :pathname "tests/tests"))
  :perform (test-op (op c) (symbol-call :application.tests :test)))

The ASD file now defines two systems: application and application.tests.

application specifies :in-order-to, which allows you to run tests on this system.

application.tests specifies :perform, which defines how to go about the testing. In this case, by calling the function test exported by the :application.tests package.

You can run tests like this:

(asdf:test-system :application)

Assuming you've defined an entry point that runs the application in standalone mode, e.g.

(defun executable-entry-point ()
  (decanter:run *app* :standalone t))

A shell script might look like this:

#!/usr/bin/env -S sbcl --script
(load "~/.sbclrc")

(require :application)

(application:executable-entry-point)

Assuming you've defined an entry point that runs the application in standalone mode, e.g.

(defun executable-entry-point ()
  (decanter:run *app* :standalone t))

Building a binary might look like this:

$ sbcl \
  --noinform --non-interactive \
  --eval "(asdf:load-system :application :force t)" \
  --eval "(require :application)" \
  --eval "(sb-ext:save-lisp-and-die #p\"application\" :toplevel #'application:executable-entry-point :executable t)"

You can run the binary like any other:

$ chmod +x application
$ ./application

You might write a function to convert database output to JSON:

(defun json (db-rows)
  (append (list :array)
          (mapcar (lambda (row)
                    (append (list :object) row))
                  db-rows)))

(defhandler index
  (:get () (json (query *db* "SELECT * FROM users"))))

Navigating to http://localhost:5000 shows this:

[
    {
        "id": 1,
        "userName": "User Name",
        "age": 25
    }
]

A transaction is a group of statements or queries that are executed atomically; it is either executed entirely, or not at all. Transactions are useful when performing complex, multi-step operations on a database that could leave the database inconsistent if they were interrupted or only partially completed.

Decanter makes it easy to perform database transactions. Simply place multiple statements or queries inside with-transaction:

(with-transaction *db*
  (statement *db*
             "CREATE TABLE users (
                  id INTEGER PRIMARY KEY,
                  user_name TEXT NOT NULL,
                  age INTEGER NULL
              )")
  (statement *db*
             "INSERT INTO users (
                  user_name,
                  age
              ) VALUES (?, ?)"
             "User Name" 25))

Applications often ship a schema.sql file that creates a relational database. You can provide a function that creates a database based on such a schema.

For example, assuming a database connection *db*:

(defun init-db ()
  (let* ((schema-str (alexandria:read-file-into-string #P"schema.sql"))
         (schema-statements (split-sequence:split-sequence #\; schema-str)))
    (with-transaction *db*
      (mapcar (lambda (statement)
                (statement *db* statement))
              schema-statements))))

This example uses the Alexandria and split-sequence libraries.

It reads the schema file into a string, then splits that string by the semicolon delimiter into different SQL statements. Then, it executes each of those statements within a transaction.

The function isn't particularly robust and won't handle comments in a SQL file.

Working with SQL strings is not always desirable. Sometimes, you might want to represent SQL using Lisp data structures, so that you could manipulate it using the full power of Lisp, rather than just string manipulation functions.

The SXQL library allows you to do this. Decanter's :decanter.ext.db package is designed to work with SXQL as seamlessly as SQL strings.

We can rewrite our example application to use SXQL:

(require :decanter)
(require :decanter.ext.db)
(require :sxql)
(use-package :decanter)
(use-package :decanter.ext.db)
(use-package :sxql)

(defparameter *db-path* ":memory:")
(defparameter *db* (connect :sqlite3 :database-name *db-path*))

(statement *db*
           (create-table :users
               ((id :type 'integer
                    :primary-key t)
                (user_name :type 'string
                           :not-null t)
                (age :type 'integer
                     :default :null))))
(statement *db*
           (insert-into :users
             (set= :user_name "User Name"
                   :age 25)))

(defurls *urls*
    '("/" index))

(defapp *app* *urls*)

(defhandler index
  (:get () (format nil "~s" (query *db* (select :* (from :users))))))

(run *app*)

Preventing SQL injection attacks is a common concern when developing web applications. Typically, SQL injection occurs when building SQL queries by concatenating strings and variables that have not been sanitized.

Decanter cleans and escapes variables automatically when binding them to SQL queries using the statement and query functions, so you don't need to worry about doing so manually.