Massimiliano Gubinelli

I record here a workflow to fix some problems with the links in the website. It can be used as a tutorial in TeXmacs scripting. The goal is to batch-modify pages. Written May 13th, 2023.

This document refers to the state of the mgubi.github.io repository checked out at commit

The goal is to change certain hyperlinks to use a user-defined macro instead, this will make easier in the future to customize the rendering or the location of the linked files.

Batch-modify pages

Scheme]

(setenv "NOTES" (url->system 
           (url-append (url-head (current-buffer)) "..")))

Scheme]

(getenv "NOTES")

"/Users/mgubi/Library/CloudStorage/Dropbox/Safe/webpages"

Scheme]

(define t (tree-import 
        (url->system "$NOTES/src/main.tm") "texmacs"))

Scheme]

(tree-export t  (url->system "$NOTES/test.tm")  "texmacs")

Scheme]

(system "diff $NOTES/test.tm $NOTES/src/main.tm")

All good (0 means success): we can read and write TeXmacs files. Let's give a look to at all hyperlinks in the file, for example.

Scheme]

(select t '(:* hlink))

We want to replace all the hlink tags with notes-link tags, so that we will be able to customize them afterwards. select returns a list of tree which should remember their positions on the document tree, so we can try to just modify them in place, one after the other. Let's try with the first:

Scheme]

(define h (car (select t '(:* hlink))))

(<tree <hlink|google scholar|http://scholar.google.ca/citations?hl=en&user=D4PR4LYAAAAJ&view_op=list_works&pagesize=100>>)

I put in a list the tree otherwise TeXmacs will try to typeset it. Let's now change the label of this tree.

Scheme]

(list (tree-assign-node! h 'notes-link))

(<tree <notes-link|google scholar|http://scholar.google.ca/citations?hl=en&user=D4PR4LYAAAAJ&view_op=list_works&pagesize=100>>)

Scheme]

(select t '(:* notes-link))

(<tree <notes-link|google scholar|http://scholar.google.ca/citations?hl=en&user=D4PR4LYAAAAJ&view_op=list_works&pagesize=100>>)

It worked! Now the documents' first hlink has been changes in notes-link. I can now proceed and change all the others.

Scheme]

(map (lambda (h) (tree-assign-node! h 'notes-link)) 
    (select t '(:* hlink)))

Scheme]

(select t '(:* hlink))

Scheme]

(tree-export t (url->system "$NOTES/test.tm")
             "texmacs")

I would like to be more selective and change only the internal links, i.e. those which do not start with http:// or https://. So let's start again. tree-ref extracts the subtrees of a given tree.

Scheme]

(define t (tree-import (url->system "$NOTES/src/main.tm")
                       "texmacs"))

Scheme]

(define l (select t '(:* hlink)))

Scheme]

(list (car l))

(<tree <hlink|google scholar|http://scholar.google.ca/citations?hl=en&user=D4PR4LYAAAAJ&view_op=list_works&pagesize=100>>)

Scheme]

(list (tree-ref (car l) 0))

(<tree google scholar>)

Scheme]

(list (tree-ref (car l) 1))

(<tree http://scholar.google.ca/citations?hl=en&user=D4PR4LYAAAAJ&view_op=list_works&pagesize=100>)

Scheme]

(tree-atomic? (tree-ref (car l) 1))

Scheme]

(tree->string (tree-ref (car l) 1))

"http://scholar.google.ca/citations?hl=en&user=D4PR4LYAAAAJ&view_op=list_works&pagesize=100"

Atomic trees are strings: I need to check if it atomic and then check if it starts with http://.

Scheme]

(string-starts? (tree->string (tree-ref (car l) 1))
                "http://")

Which ideed it does. I can now filter all the similar trees and change only those.

Scheme]

(define l1 (filter (lambda (h) 
   (not (and (tree-atomic? (tree-ref h 1))
       (or  (string-starts? (tree->string (tree-ref h 1))
                           "http://")  
        (string-starts? (tree->string (tree-ref h 1))
                        "https://")))))
      l))

Scheme]

(map (lambda (h) (tree-assign-node! h 'notes-link)) l1)

I would like also to normalize the filenames to avoid ./ since it is redundant. Note that now l1 has changed.

So I can iterate again and change the second child to drop ./ whenever needed. To do this we use tree-set! which replaces in place a given tree relative to a base tree.

Scheme]

(map (lambda (h)
       (if (and (tree-atomic? (tree-ref h 1)) 
                (string-starts? (tree->string (tree-ref h 1))
                                "./"))
            (tree-set! h 1 (string-drop 
                            (tree->string (tree-ref h 1)) 2))))
      (select t '(:* notes-link)))

(<tree curriculum-vitae.tm> #<unspecified> #<unspecified> <tree programming.tm> #<unspecified> <tree list-articles.tm> #<unspecified> #<unspecified>)

Scheme]

(select t '(:* notes-link))

Yes, it did. We wrap now all the process in a function which operate on a given tree.

Scheme]

(define (handle-links t) (let* 
  ((l (select t '(:* hlink)))
  (l1 (filter (lambda (h) 
   (not (and (tree-atomic? (tree-ref h 1))
       (or  (string-starts? (tree->string (tree-ref h 1))
                           "http://")  
        (string-starts? (tree->string (tree-ref h 1))
                        "https://")))))
      l)))
  (map (lambda (h)
       (tree-assign-node! h 'notes-link)
       (if (and (tree-atomic? (tree-ref h 1)) 
                (string-starts? (tree->string (tree-ref h 1))
                                "./"))
           (tree-set! h 1 (string-drop 
                            (tree->string (tree-ref h 1)) 2))))
      l1)
  ))

Scheme]

(define t (tree-import (url->system "$NOTES/src/main.tm")
                       "texmacs"))

Scheme]

(handle-links t)

(<tree curriculum-vitae.tm> #<unspecified> #<unspecified> <tree programming.tm> #<unspecified> <tree list-articles.tm> #<unspecified> #<unspecified>)

Scheme]

(select t '(:* (:or hlink notes-link)))

Improving references to stored files

We want to perform also another batch change, and replace all the links into the store with a new tag notes-store.

Scheme]

(define t (tree-import (url->system "$NOTES/src/people.tm")
                       "texmacs"))

Scheme]

(define l (filter (lambda (h) 
              (string-contains?  (tree->string (tree-ref h 1))
                                 "store/")) 
                (select t '(:* hlink)))))

We check that an hyperlink has store/ as sublink and we preform the substitions in this case.

Scheme]

(string-contains "../store/master-thesis-barashkov.pdf" "store/")

Scheme]

(string-drop "../store/master-thesis-barashkov.pdf" 9)

"master-thesis-barashkov.pdf"

Scheme]

(for-each (lambda (h) 
     (tree-assign-node! h 'notes-store) 
     (let* ((s (tree->string (tree-ref h 1)))
            (s1 (string-drop s 
                    (+ 6 (string-contains s "store/")))))
       (tree-set! h 1 s1))) 
          l)

Ok. We wrap all the process in an handy function. (which has to be applied after handle-links)

Scheme]

(define (handle-notes-store t)
  (with l (filter (lambda (h) (string-contains?  
                  (tree->string (tree-ref h 1)) "store/")) 
                (select t '(:* notes-link)))
(for-each (lambda (h) 
     (tree-assign-node! h 'notes-store) 
     (let* ((s (tree->string (tree-ref h 1)))
            (s1 (string-drop s 
                  (+ 6 (string-contains s "store/")))))
       (tree-set! h 1 s1)))
    l)))

Scheme]

 (handle-links t)

(#<unspecified> #<unspecified> #<unspecified> #<unspecified> #<unspecified> #<unspecified> #<unspecified> #<unspecified> #<unspecified>)

Scheme]

(handle-notes-store t)

Scheme]

(select t '(:* notes-store))

Putting all together

Now we aim to batch-process all the files in a given directory. We gather all the files in the $NOTES/src directory, removing the prefix to make easier later manipulations.

Scheme]

(define u1 (url-append (url-append "$NOTES/src" 
                                   (url-any))  
                      "*.tm"))

<url /Users/mgubi/Library/CloudStorage/Dropbox/Safe/webpages/src/{**}/*.tm>

Scheme]

(define files (url->list (url-delta "$NOTES/src/dummy" 
                (url-expand (url-complete u1 "fr")))))

Scheme]

(list-take files 5)

(<url curriculum-vitae.tm> <url events.tm> <url list-articles.tm> <url main.tm> <url old-publications.tm>)

Scheme]

(url-directory? (url-append (url-append "$NOTES/src" 
                                    (car files)) 
                         (url-parent)))

Scheme]

(url-head (car files))

Scheme]

(define (make-dir-tree url)
  (when (!= url (system->url "$NOTES/src2")) 
      (make-dir-tree (url-expand 
                      (url-append url (url-parent))))
      (when (not (url-exists? url))     
        (system-mkdir url)
        (system-1 "chmod a+x" url))))

Scheme]

(define dirs (ahash-set->list (list->ahash-set 
          (map (lambda (f) (url-head 
                          (url-append "$NOTES/src2" f)))
               files))))

Scheme]

(for-each make-dir-tree dirs)

This created the right arborescence. We are ready now to convert each file in files.

Scheme]

(define (proc-file url)
  (define t (tree-import (url->system 
                          (url-append "$NOTES/src" url))  
                         "texmacs"))
  (handle-links t)
  (handle-notes-store t)
  (tree-export t (url->system 
                  (url-append "$NOTES/src2" url)) 
               "texmacs"))

Scheme]

(proc-file "main.tm")

Scheme]

(map proc-file files)

(#f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f #f)

Ok. After checking that there are no problems, we can copy the modified files from src2 to src.

Scripting webpages

Batch-modify pages

Improving references to stored files

Putting all together