curl-friendly way to upload files #43

New issue

Open

opened 2024-11-27 15:46:55 +01:00 by luna · 1 comment

luna commented

2024-11-27 15:46:55 +01:00

Member

running curl -fsS --user user:pass -F .file=@file.txt https://once.schlittermann.de to send the appropriate POST request to the once server is easy enough, however, parsing the returned HTML is not.

I made it work using htmlq (❎ in Debian repos), sed, and yq (✅ in Debian repos)

once-upload() {
    curl -fsS \
        --user "$(pass private/$USER/once)" \
        -F .file=@"${1:-filename needed}" \
        https://once.schlittermann.de/ \
    | htmlq -b https://once.schlittermann.de 'tr.listing td:nth-child(1) a, tr.listing td:nth-child(3)' \
    | sed -e '1i<wrap>' -e '$a</wrap>' \
    | xq -r '[.wrap.a, .wrap.td] | transpose | sort_by(.[1]) | last[0]."@href"'
}

The xq (xml wrapper around jq) pipeline is necessary to find the last uploaded file (by using the fact that ISO format datetimes are lexicographically sortable) since it's the only way to figure out which foo.txt you just uploaded, if there are multiple files with the same name.

Maybe once could detect the curl user agent and respond with a basic JSON response or maybe even just a plain download link.

running `curl -fsS --user user:pass -F .file=@file.txt https://once.schlittermann.de` to send the appropriate POST request to the once server is easy enough, however, parsing the returned HTML is not. I made it work using [htmlq](https://github.com/mgdm/htmlq) (❎ in Debian repos), sed, and [yq](https://github.com/kislyuk/yq) (✅ in Debian repos) ```bash once-upload() { curl -fsS \ --user "$(pass private/$USER/once)" \ -F .file=@"${1:-filename needed}" \ https://once.schlittermann.de/ \ | htmlq -b https://once.schlittermann.de 'tr.listing td:nth-child(1) a, tr.listing td:nth-child(3)' \ | sed -e '1i<wrap>' -e '$a</wrap>' \ | xq -r '[.wrap.a, .wrap.td] | transpose | sort_by(.[1]) | last[0]."@href"' } ``` The `xq` (xml wrapper around `jq`) pipeline is necessary to find the last uploaded file (by using the fact that ISO format datetimes are lexicographically sortable) since it's the only way to figure out which `foo.txt` you just uploaded, if there are multiple files with the same name. Maybe once could detect the `curl` user agent and respond with a basic JSON response or maybe even just a plain download link.

heiko commented

2025-05-23 10:06:15 +02:00

Owner

Suggestion:

extend the once server to return parseable output (by checking the client's "Accept" header for e.g. "Accept: application/json"
create a once-client (I think, there's work in progress already) which makes use of the parseable output

Suggestion: - extend the once server to return parseable output (by checking the client's "Accept" header for e.g. "Accept: application/json" - create a once-client (I think, there's work in progress already) which makes use of the parseable output