diff --git a/README.md b/README.md
index e9a7bf4..998ab11 100644
--- a/README.md
+++ b/README.md
@@ -3,7 +3,7 @@
CSVtoTable converts CSV, TSV, and Excel files into interactive HTML tables.
- Single native binary with embedded frontend assets
-- Standalone HTML output that works offline — data, styles, and scripts all inlined
+- Standalone HTML output that works offline — data, styles, and scripts all inlined by default
- CSV, TSV, and Excel (`.xlsx`) input, gzip archives included
- Local files, URLs, standard input, or several files combined into one table
- BOM and UTF-16 detected automatically; `--encoding`, `--delimiter`, `--quotechar` for the rest
@@ -13,6 +13,8 @@ CSVtoTable converts CSV, TSV, and Excel files into interactive HTML tables.
- Five colour themes, switchable in the page or fixed with `--theme`
- Themes are just CSS variables — define your own with `--css` and it joins the picker
- `--css` and `--js` inline your own stylesheet and script, with the live table API exposed
+- Self-unpacking output: the frontend ships gzipped, roughly halving every file
+- `--split` into separately cacheable assets instead, or `--serve` a preview over HTTP
- Mobile-responsive layout
@@ -37,9 +39,15 @@ csvtotable sales.xlsx sales.html
# Fetch CSV directly from a URL
csvtotable https://raw.githubusercontent.com/vividvilla/csvtotable/master/demo/meteorite-landings-1.csv meteorites.html
-# Open a temporary page in the default browser
+# Build and open the page on a local HTTP server
csvtotable data.csv --serve
+# Serve it on a port you choose
+csvtotable data.csv --serve :8080
+
+# Write a directory of separate files instead of one page
+csvtotable data.csv site/ --split
+
# Add a title and generate headers for headerless data
csvtotable data.csv data.html --title "Sales" --no-header
@@ -64,11 +72,12 @@ csvtotable data.csv data.html --css brand.css --js setup.js
# Read stdin and write stdout
curl -L https://example.com/data.csv | csvtotable - - > data.html
-
-# Explore all the available options
-csvtotable --help
```
+Run `csvtotable --help` for all options or `csvtotable --version` for the version.
+For compatibility with version 2, `--caption`, `--display-length`, `--pagination`,
+and `--export` still work; the last two disable those features.
+
## Install
### uvx, pipx, or pip
@@ -116,7 +125,63 @@ on your `PATH`.
Prebuilt binaries support Linux x86-64/ARM64, macOS 12+ x86-64/Apple Silicon,
and Windows 10+ x86-64.
-### Styling
+## Output
+
+### Size
+
+The frontend script is gzipped and base64'd into the page, cutting an otherwise
+empty file from about 260KB to 145KB. Unpacking needs `DecompressionStream`
+(Chrome 103+, Firefox 113+, Safari 16.4+); older browsers get a message saying
+so, and `--no-compress` inlines readable source instead. The stylesheet stays
+uncompressed either way, so the page is styled at first paint.
+
+### Separate files
+
+`--split` writes the output path as a directory instead of a single page:
+
+```
+site/
+ index.html the page, a couple of KB
+ csvtotable.9bf9d6348cd4.css the stylesheet
+ csvtotable.47304c323fd7.js the frontend
+ data.50248c1cfaed.js the rows
+```
+
+References are relative, so the directory serves from any path, and the browser
+caches the frontend like any other asset — behind a gzipping server the demo data
+costs roughly 105KB the first time and 31KB on a revisit.
+
+Filenames carry a content hash: an unchanged rerun keeps the URL and the cache
+hit, while changed rows get a new one that cannot be served stale. Superseded
+files are left in place for you to clear out. `index.html` keeps its name, so its
+freshness is up to whatever serves it.
+
+Nothing needs `fetch`, so `index.html` still opens from disk. `--css` and `--js`
+stay inline — a `--css` theme has to be, for the theme picker to find it over
+`file://`. Compression does not apply here; caching does that job.
+
+### Serving
+
+`--serve` builds the page into a temporary directory, serves it over HTTP, and
+opens a browser there. It takes an optional `[HOST]:PORT`:
+
+```sh
+csvtotable data.csv --serve # a random loopback port
+csvtotable data.csv --serve :8080 # port 8080 on loopback
+csvtotable data.csv --serve 0.0.0.0:8080 # every interface
+csvtotable data.csv --serve --split # each asset served separately
+```
+
+An empty host binds loopback; exposing the data on the network takes writing the
+host out, and prints a warning. The address is printed either way, so the page is
+reachable if no browser opens.
+
+Responses carry `Cache-Control: no-store`, since the directory is rebuilt each
+run onto a port the kernel reuses and caching it would mix runs together. The
+directory is removed on exit. A `--split` directory you deploy yourself caches
+normally.
+
+## Styling
The page is plain semantic HTML, and every element CSVtoTable owns carries a
`csvtotable-` class. These are the stable hooks:
@@ -132,6 +197,7 @@ The page is plain semantic HTML, and every element CSVtoTable owns carries a
| `.csvtotable-filters` | active-filter chip row |
| `.csvtotable-chip` | one active filter, with `-key`, `-value`, and `-remove` parts |
| `.csvtotable-clear` | the "clear all" control |
+| `.csvtotable-error` | shown only when the page cannot unpack itself |
A theme is nothing but a block of colour variables. Every rule in the
stylesheet reads them, so a new theme is a copy of one block with different
@@ -164,52 +230,45 @@ beginning `dt-` come from DataTables and may change when it is upgraded.
### Custom CSS and JavaScript
`--css` and `--js` inline a stylesheet and a script into the page, keeping the
-output a single self-contained file. Both take a file path:
+output self-contained. Both take a file path; a leading `@` is accepted but
+means nothing here:
```sh
csvtotable data.csv data.html --css brand.css --js setup.js
```
-A leading `@` is accepted too, for symmetry with `--description`, but means
-nothing here: `--css @brand.css` and `--css brand.css` are the same.
-
-Placement is what makes them useful. The stylesheet goes last in ``, after
-the built-in one, so a single class selector overrides anything above without
-`!important`. The script goes last in ``, after the table is built, and
-`CsvToTable.table` holds the live [DataTables API](https://datatables.net/reference/api/)
-instance:
+Placement is the point. The stylesheet goes last in ``, so a single class
+selector overrides the built-in one without `!important`. The script goes last
+in ``, after the table is built, with `CsvToTable.table` holding the live
+[DataTables API](https://datatables.net/reference/api/) instance:
```js
CsvToTable.table.order([2, "desc"]).draw(); // sort by the third column
CsvToTable.table.column(0).visible(false); // hide the first column
```
-The table's height is fitted on the next animation frame, so a script that
-measures layout should wrap the read in `requestAnimationFrame`. Column widths
-and the scroll height are set as inline styles, which a stylesheet cannot
-override — use `--height` for that.
+The table's height is fitted on the next animation frame, so wrap layout reads
+in `requestAnimationFrame`; column widths and scroll height are inline styles a
+stylesheet cannot override — use `--height`. In a compressed page the script is
+parked in an inert `\n"
- }
themeAttribute := ""
if cli.Theme != "" && cli.Theme != "auto" {
@@ -500,7 +671,10 @@ func convert(cli options, destination io.Writer) error {
if err != nil {
return err
}
- output := bufio.NewWriter(destination)
+ stylesheetHTML := "\n"
+ if split != nil {
+ stylesheetHTML = `\n"
+ }
headers := []string{}
expectedColumns := -1
started := false
@@ -510,7 +684,14 @@ func convert(cli options, destination io.Writer) error {
if err != nil {
return err
}
- _, err = fmt.Fprintf(output, "\n\n\n\n\n%s\n\n%s\n\n\n%s%s\n
\n\n\n\n\n\n%s\n\n", optionsJSON, inlineScript(tableJS), scriptHTML); err != nil {
+ if split != nil {
+ if _, err := io.WriteString(data, "]};\n"); err != nil {
+ return err
+ }
+ // The data file is complete, so its hash — and therefore its name — is
+ // settled and the page can link to it.
+ if err := data.Flush(); err != nil {
+ return err
+ }
+ } else if _, err := io.WriteString(page, "]}\n"); err != nil {
return err
}
- return output.Flush()
+ scriptsHTML, err := scriptsMarkup(cli.Compress, customJS, split)
+ if err != nil {
+ return err
+ }
+ _, err = fmt.Fprintf(page, "\n%s\n\n", optionsJSON, scriptsHTML)
+ return err
+}
+
+// bootstrapFormat builds the table once the frontend bundle is in scope. Every
+// path ends by running it; they differ only in where the rows come from.
+const bootstrapFormat = `CsvToTable.setupTheme("#csvtotable-theme");CsvToTable.table=CsvToTable.createCsvTable("#csvtotable-table",%s,JSON.parse(document.getElementById("csvtotable-options").textContent));`
+
+var (
+ bootstrap = fmt.Sprintf(bootstrapFormat, `JSON.parse(document.getElementById("csvtotable-data").textContent)`)
+ splitBootstrap = fmt.Sprintf(bootstrapFormat, "window.csvtotableData")
+)
+
+// indexFile is the only fixed name --split writes: everything it links to
+// carries a hash of its contents, so a regenerated directory can never serve a
+// stale asset against a fresh page. The entry point has to stay put for the
+// URL to keep working, which leaves its freshness to the server, as with any
+// static site.
+const indexFile = "index.html"
+
+// splitAssets are the files a --split page links to. The data file is named
+// last: its hash is only known once every row has been written.
+type splitAssets struct {
+ css string
+ js string
+ dataName func() string
+}
+
+// hashedName inserts a content digest before the extension, so unchanged
+// assets keep their URL across runs and changed ones cannot be mistaken for
+// them.
+func hashedName(stem, extension, content string) string {
+ digest := sha256.Sum256([]byte(content))
+ return fmt.Sprintf("%s.%s%s", stem, hex.EncodeToString(digest[:])[:12], extension)
+}
+
+// inflater unpacks the gzipped bundle, builds the table, then runs whatever
+// --js supplied. Appending ` + "\n" +
+ `` + "\n" +
+ "\n"
+ if custom != "" {
+ markup += "\n"
+ }
+ return markup, nil
+ }
+ if !compress {
+ markup := "\n\n"
+ if custom != "" {
+ markup += "\n"
+ }
+ return markup, nil
+ }
+ packed, err := packAsset(tableJS)
+ if err != nil {
+ return "", err
+ }
+ markup := `\n"
+ if custom != "" {
+ markup += `\n"
+ }
+ return markup + "\n", nil
+}
+
+// packAsset gzips and base64-encodes an asset for the inflater. Base64 has no
+// HTML-significant characters, so the result needs none of the escaping raw
+// source does.
+func packAsset(source string) (string, error) {
+ var packed bytes.Buffer
+ writer, err := gzip.NewWriterLevel(&packed, gzip.BestCompression)
+ if err != nil {
+ return "", err
+ }
+ if _, err := io.WriteString(writer, source); err != nil {
+ return "", err
+ }
+ if err := writer.Close(); err != nil {
+ return "", err
+ }
+ return base64.StdEncoding.EncodeToString(packed.Bytes()), nil
+}
+
+// convertDirectory writes index.html and its assets into dir as separate
+// files, referenced by relative path. Opening index.html from disk still works
+// — and ") {
+ t.Error("custom JS runs on its own, before the bundle is unpacked")
+ }
+
+ // Decoding the blob has to give back the bundle byte for byte.
+ encoded := page[blob+len(`")]
+ raw, err := base64.StdEncoding.DecodeString(encoded)
+ if err != nil {
+ t.Fatalf("bundle is not valid base64: %v", err)
+ }
+ reader, err := gzip.NewReader(bytes.NewReader(raw))
+ if err != nil {
+ t.Fatalf("bundle is not gzip: %v", err)
+ }
+ unpacked, err := io.ReadAll(reader)
+ if err != nil {
+ t.Fatal(err)
+ }
+ if string(unpacked) != tableJS {
+ t.Error("the unpacked bundle differs from the embedded one")
+ }
+}
+
+func TestSplitOutput(t *testing.T) {
+ directory := t.TempDir()
+ input := filepath.Join(directory, "input.csv")
+ if err := os.WriteFile(input, []byte("city,temperature\nPune,29\nKochi,31\n"), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ target := filepath.Join(directory, "site")
+ cli, err := parseArgs([]string{input, target, "--split", "--title", "Weather"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := run(cli); err != nil {
+ t.Fatal(err)
+ }
+
+ read := func(name string) string {
+ content, err := os.ReadFile(filepath.Join(target, name))
+ if err != nil {
+ t.Fatalf("%s: %v", name, err)
+ }
+ return string(content)
+ }
+ jsName := hashedName("csvtotable", ".js", tableJS)
+ cssName := hashedName("csvtotable", ".css", tableCSS)
+ if read(jsName) != tableJS || read(cssName) != tableCSS {
+ t.Error("the written assets differ from the embedded ones")
+ }
+
+ // The page must reference its assets by relative path, so that it works
+ // from a subdirectory of whatever ends up serving it, and every reference
+ // must carry a content hash so a redeploy cannot be served stale.
+ page := read(indexFile)
+ for _, want := range []string{
+ ``,
+ ``,
+ "window.csvtotableData",
+ } {
+ if !strings.Contains(page, want) {
+ t.Errorf("index.html is missing %q", want)
+ }
+ }
+ dataRef := regexp.MustCompile(``).FindStringSubmatch(page)
+ if dataRef == nil {
+ t.Fatalf("index.html has no hashed data reference:\n%s", page)
+ }
+ dataName := dataRef[1]
+ // Nothing that belongs in a separate file may also be inlined, or the
+ // caching the mode exists for is wasted.
+ for _, unwanted := range []string{"DataTables 3.0.2", "--ct-accent", "csvtotable-bundle", `"rows":[`} {
+ if strings.Contains(page, unwanted) {
+ t.Errorf("index.html still inlines %q", unwanted)
+ }
+ }
+ if len(page) > 4096 {
+ t.Errorf("index.html is %d bytes; it should hold no payload", len(page))
+ }
+
+ // The rows have to survive the trip into a separate script.
+ data := read(dataName)
+ encoded, ok := strings.CutPrefix(strings.TrimSpace(data), "window.csvtotableData=")
+ if !ok {
+ t.Fatalf("data.js does not assign the payload: %.60q", data)
+ }
+ var payload struct {
+ Headers []string `json:"headers"`
+ Rows [][]string `json:"rows"`
+ }
+ if err := json.Unmarshal([]byte(strings.TrimSuffix(encoded, ";")), &payload); err != nil {
+ t.Fatalf("data.js is not valid JSON: %v", err)
+ }
+ if len(payload.Headers) != 2 || len(payload.Rows) != 2 || payload.Rows[1][0] != "Kochi" {
+ t.Errorf("unexpected payload: %+v", payload)
+ }
+
+ // A rerun overwrites its own files and must not need a prompt when told.
+ cli, err = parseArgs([]string{input, target, "--split", "--overwrite"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if err := run(cli); err != nil {
+ t.Fatalf("rerunning with --overwrite failed: %v", err)
+ }
+
+ // Different rows must land on a different URL, or a browser holding the
+ // old data.js serves it against the new page.
+ if err := os.WriteFile(input, []byte("city,temperature\nPune,29\nKochi,31\nDelhi,18\n"), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ if err := run(cli); err != nil {
+ t.Fatal(err)
+ }
+ if again := read(indexFile); strings.Contains(again, dataName) {
+ t.Errorf("changed rows kept the data URL %q", dataName)
+ }
+
+ // A conversion that fails must leave the served directory as it was.
+ before := read(indexFile)
+ broken := cli
+ broken.CSS = filepath.Join(directory, "missing.css")
+ if err := convertDirectory(broken, target); err == nil {
+ t.Error("a missing --css file was accepted")
+ }
+ if after := read(indexFile); after != before {
+ t.Error("a failed conversion left the previous index.html damaged")
+ }
+ entries, err := os.ReadDir(target)
+ if err != nil {
+ t.Fatal(err)
+ }
+ for _, entry := range entries {
+ if strings.HasPrefix(entry.Name(), ".csvtotable-") {
+ t.Errorf("a failed conversion left the staging file %q behind", entry.Name())
+ }
+ }
+}
+
+func TestPreviewIsNotCached(t *testing.T) {
+ directory := t.TempDir()
+ if err := os.WriteFile(filepath.Join(directory, indexFile), []byte("first"), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ handler := previewHandler(directory)
+
+ // --serve rebuilds its directory every run onto a port the kernel reuses,
+ // so anything the browser keeps is a stale asset waiting to be mixed into
+ // a later page.
+ recorder := httptest.NewRecorder()
+ handler.ServeHTTP(recorder, httptest.NewRequest(http.MethodGet, "/", nil))
+ if got := recorder.Header().Get("Cache-Control"); got != "no-store" {
+ t.Errorf("Cache-Control is %q, want no-store", got)
+ }
+ if recorder.Body.String() != "first" {
+ t.Errorf("served %q, want the file contents", recorder.Body.String())
+ }
+
+ if err := os.WriteFile(filepath.Join(directory, indexFile), []byte("second"), 0o644); err != nil {
+ t.Fatal(err)
+ }
+ recorder = httptest.NewRecorder()
+ request := httptest.NewRequest(http.MethodGet, "/", nil)
+ request.Header.Set("If-Modified-Since", time.Now().UTC().Format(http.TimeFormat))
+ handler.ServeHTTP(recorder, request)
+ if recorder.Body.String() != "second" {
+ t.Errorf("a rewritten file served %q; the preview must not go stale", recorder.Body.String())
+ }
+}
+
+func TestServeAddress(t *testing.T) {
+ // --serve takes an optional value, which urfave/cli has no notion of, so a
+ // bare --serve must not swallow the argument after it.
+ forms := []struct {
+ args []string
+ serve bool
+ address string
+ inputs int
+ }{
+ {args: []string{"in.csv", "--serve"}, serve: true, inputs: 1},
+ {args: []string{"--serve", "in.csv"}, serve: true, inputs: 1},
+ {args: []string{"-s", "in.csv"}, serve: true, inputs: 1},
+ {args: []string{"--serve", ":8080", "in.csv"}, serve: true, address: ":8080", inputs: 1},
+ {args: []string{"--serve=127.0.0.1:8080", "in.csv"}, serve: true, address: "127.0.0.1:8080", inputs: 1},
+ {args: []string{"--serve", "[::1]:8080", "in.csv"}, serve: true, address: "[::1]:8080", inputs: 1},
+ {args: []string{"in.csv", "out.html"}, serve: false, inputs: 1},
+ }
+ // A -s that belongs to another flag is that flag's value, not this one.
+ guarded, err := parseArgs([]string{"--title", "-s", "in.csv", "out.html"})
+ if err != nil {
+ t.Fatal(err)
+ }
+ if guarded.Title != "-s" || guarded.Serve || len(guarded.InputFiles) != 1 {
+ t.Errorf("--title -s was rewritten: title=%q serve=%v inputs=%v",
+ guarded.Title, guarded.Serve, guarded.InputFiles)
+ }
+ for _, form := range forms {
+ cli, err := parseArgs(form.args)
+ if err != nil {
+ t.Errorf("%v: %v", form.args, err)
+ continue
+ }
+ if cli.Serve != form.serve || cli.Address != form.address || len(cli.InputFiles) != form.inputs {
+ t.Errorf("%v: serve=%v address=%q inputs=%v; want %v, %q, %d",
+ form.args, cli.Serve, cli.Address, cli.InputFiles, form.serve, form.address, form.inputs)
+ }
+ }
+
+ // An empty host binds loopback: putting the data on the network should
+ // take more than leaving the host off.
+ targets := map[string]string{
+ "": "127.0.0.1:0",
+ ":8080": "127.0.0.1:8080",
+ "localhost:8080": "localhost:8080",
+ "0.0.0.0:8080": "0.0.0.0:8080",
+ "[::1]:8080": "[::1]:8080",
+ }
+ for address, want := range targets {
+ got, err := listenTarget(address)
+ if err != nil || got != want {
+ t.Errorf("listenTarget(%q) = %q, %v; want %q", address, got, err, want)
+ }
+ }
+ for _, bad := range []string{"8080", "nonsense", ":99999", "host:port"} {
+ if _, err := listenTarget(bad); err == nil {
+ t.Errorf("listenTarget(%q) was accepted", bad)
+ }
+ }
+
+ for bind, want := range map[string]bool{
+ "127.0.0.1:80": true, "localhost:80": true, "[::1]:80": true,
+ "0.0.0.0:80": false, "10.0.0.4:80": false,
+ } {
+ if got := loopbackOnly(bind); got != want {
+ t.Errorf("loopbackOnly(%q) = %v, want %v", bind, got, want)
+ }
+ }
+}
diff --git a/table/src/table.css b/table/src/table.css
index d43aea6..48a92d6 100644
--- a/table/src/table.css
+++ b/table/src/table.css
@@ -171,6 +171,17 @@ code {
.csvtotable-description ol { padding-left: 1.1rem; }
.csvtotable-description a { color: var(--ct-accent); }
+/* Shown only when the compressed bundle cannot be inflated, so the reader gets
+ a reason instead of an empty page. */
+.csvtotable-error {
+ margin: 0 0 1.1rem;
+ padding: .7rem .9rem;
+ border-left: 3px solid var(--ct-accent);
+ background: var(--ct-wash);
+ color: var(--ct-muted);
+ font-size: var(--ct-text-base);
+}
+
/* Toolbar ------------------------------------------------------------- */
div.dt-container { color: var(--ct-ink); font-size: var(--ct-text-base); }