Paul Legato

I wanted to parse some externally-generated and malformed HTML, so naturally I went to the short and sweet clojure.xml/parse function. I got a nasty error:

error: java.io.IOException: Server returned HTTP response code: 503 for URL: http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd

It seems that the W3C blocked access to the DTDs two years ago, but Java still tries to load them by default anyway. The following allows clojure.xml to run without checking external DTDs:

(defn startparse-sax-non-validating [s ch]
  (.. (doto (. javax.xml.parsers.SAXParserFactory (newInstance))
       (.setValidating false)
       (.setFeature "http://apache.org/xml/features/nonvalidating/load-dtd-grammar" false)
       (.setFeature "http://apache.org/xml/features/nonvalidating/load-external-dtd" false)
       (.setFeature "http://xml.org/sax/features/validation" false)
       (.setFeature "http://xml.org/sax/features/external-general-entities" false)
       (.setFeature "http://xml.org/sax/features/external-parameter-entities" false))

       (newSAXParser) (parse s ch))))

Then you can simply (xml/parse "sourcefile.xml" startparse-sax-non-validating). This is not the ideal solution — ideally, we want to use a locally cached DTD — but it works well enough for one-off code. Read on for further information. Continue Reading »

Popularity: 36% [?]

And it came to pass, that Goldman’s then-chief overlord was order’d to parade full 7 days and 7 nights through the streets of lower Manhattan, clad in naught save his drawers, bearing tether’d to heavy chains every SEC filing that Goldman hath wrought for 10 years previous; while the peasants of the borough were to gather round to hurl rotting vegetables thence, and cry, “Lo! How the mighty have fallen!”

And only then, once the indignity were suffer’d in full, might he retreat to his penthouse to recover, or to his yacht, or to the coffers of his chalet in southern France, or to his Bahamas estate countinghouse, or to any other sequester’d location whatsoever.

Popularity: 3% [?]

Psyleron sells a hardware random number generator and associated software package for experimentation on the interaction of consciousness and randomness. No, don’t laugh. Princeton Engineering Anomalies Research conducted decades of methodologically rigorous research into the nature of randomness. They concluded that deliberate conscious intention can produce a small but statistically significant and reproducible effect on the outcome of stochastic processes. Psyleron is a commercial offshoot of PEAR.

The Psyleron system, unfortunately, costs hundreds of dollars and only runs on Windows, so I wrote some quick Clojure to do experimentation with randomness, using /dev/random as the source. (It appears that /dev/random on Mac OS X does not provide the same quality guarantees as it does on Linux, but it’s fine for a prototype and the implementation allows easy substitution of another randomness source later.) Continue Reading »

Popularity: 24% [?]

SQL WHERE and HAVING clause strings can be rendered from neat, structured S-expressions with this simple Clojure macro:

(defmacro sql-expand
  "Transforms nested s-expressions into SQL, for use in WHERE or HAVING clauses.

e.g.: (sql-expand (and (> foo 3)
                       (< bar 4)))

     -> '(foo > 3 AND bar < 4)'

 Nested clauses:

      (sql-expand (or (and (> foo 3) (< bar 4))
                      (> baz 6)))

      -> '((foo > 3 AND bar < 4) OR baz > 6)'

 Embedded arbitrary SQL:
         (sql-expand (and (> foo 3)
                          (< bar \"(SELECT max(foo) + 10 FROM bar)\")))
      -> '(foo > 3 AND bar < (SELECT max(foo) + 10 FROM bar))'

"
  [form]
  (let [head (first form)]
    (if (includes? '(and or) head)
      `(str "(" (sql-expand ~(second form)) " "
            ~(.toUpperCase (str head)) " "
            (sql-expand ~(last form)) ")")
      (str (second form) " " head " " (last form)))))

Continue Reading »

Popularity: 30% [?]