EuAndreh's websiteTILhttps://euandre.org/en/til/feed.git.xml2026-09-13T12:14:51+00:00EuAndreheu@euandre.org
Storing CI data on Git notes
2020-11-30T00:00:00+00:00
https://euandre.org/en/til/2020/11/30/git-notes-ci.html
EuAndreh
eu@euandre.org
<div class="paragraph">
<p>Extending the bare bones CI server I’ve <a href="../12/diy-nix-bash-ci.html">talked about before</a>,
divoplade on Freenode suggested storing CI artifacts on <a href="https://git-scm.com/docs/git-notes">Git
notes</a>, such as tarballs, binaries, logs, <em>etc</em>.</p>
</div>
<div class="paragraph">
<p>Extending the bare bones CI server I’ve <a href="../12/diy-nix-bash-ci.html">talked about before</a>,
divoplade on Freenode suggested storing CI artifacts on <a href="https://git-scm.com/docs/git-notes">Git
notes</a>, such as tarballs, binaries, logs, <em>etc</em>.</p>
</div>
<div class="paragraph">
<p>I’ve written a small script that will put log files and CI job data on Git notes, and make it visible on
the porcelain log. It is a simple extension of the previous article:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B1-L1"
href="#B1-L1">1</a>
<a class="code-line-anchor"
id="B1-L2"
href="#B1-L2">2</a>
<a class="code-line-anchor"
id="B1-L3"
href="#B1-L3">3</a>
<a class="code-line-anchor"
id="B1-L4"
href="#B1-L4">4</a>
<a class="code-line-anchor"
id="B1-L5"
href="#B1-L5">5</a>
<a class="code-line-anchor"
id="B1-L6"
href="#B1-L6">6</a>
<a class="code-line-anchor"
id="B1-L7"
href="#B1-L7">7</a>
<a class="code-line-anchor"
id="B1-L8"
href="#B1-L8">8</a>
<a class="code-line-anchor"
id="B1-L9"
href="#B1-L9">9</a>
<a class="code-line-anchor"
id="B1-L10"
href="#B1-L10">10</a>
<a class="code-line-anchor"
id="B1-L11"
href="#B1-L11">11</a>
<a class="code-line-anchor"
id="B1-L12"
href="#B1-L12">12</a>
<a class="code-line-anchor"
id="B1-L13"
href="#B1-L13">13</a>
<a class="code-line-anchor"
id="B1-L14"
href="#B1-L14">14</a>
<a class="code-line-anchor"
id="B1-L15"
href="#B1-L15">15</a>
<a class="code-line-anchor"
id="B1-L16"
href="#B1-L16">16</a>
<a class="code-line-anchor"
id="B1-L17"
href="#B1-L17">17</a>
<a class="code-line-anchor"
id="B1-L18"
href="#B1-L18">18</a>
<a class="code-line-anchor"
id="B1-L19"
href="#B1-L19">19</a>
<a class="code-line-anchor"
id="B1-L20"
href="#B1-L20">20</a>
<a class="code-line-anchor"
id="B1-L21"
href="#B1-L21">21</a>
<a class="code-line-anchor"
id="B1-L22"
href="#B1-L22">22</a>
<a class="code-line-anchor"
id="B1-L23"
href="#B1-L23">23</a>
<a class="code-line-anchor"
id="B1-L24"
href="#B1-L24">24</a>
<a class="code-line-anchor"
id="B1-L25"
href="#B1-L25">25</a>
<a class="code-line-anchor"
id="B1-L26"
href="#B1-L26">26</a>
<a class="code-line-anchor"
id="B1-L27"
href="#B1-L27">27</a>
<a class="code-line-anchor"
id="B1-L28"
href="#B1-L28">28</a>
<a class="code-line-anchor"
id="B1-L29"
href="#B1-L29">29</a>
<a class="code-line-anchor"
id="B1-L30"
href="#B1-L30">30</a>
<a class="code-line-anchor"
id="B1-L31"
href="#B1-L31">31</a>
<a class="code-line-anchor"
id="B1-L32"
href="#B1-L32">32</a>
<a class="code-line-anchor"
id="B1-L33"
href="#B1-L33">33</a>
<a class="code-line-anchor"
id="B1-L34"
href="#B1-L34">34</a>
<a class="code-line-anchor"
id="B1-L35"
href="#B1-L35">35</a>
<a class="code-line-anchor"
id="B1-L36"
href="#B1-L36">36</a>
<a class="code-line-anchor"
id="B1-L37"
href="#B1-L37">37</a>
<a class="code-line-anchor"
id="B1-L38"
href="#B1-L38">38</a>
<a class="code-line-anchor"
id="B1-L39"
href="#B1-L39">39</a>
<a class="code-line-anchor"
id="B1-L40"
href="#B1-L40">40</a></pre>
</td>
<td>
<pre>#!/usr/bin/env bash
set -Eeuo pipefail
set -x
PREFIX='/srv/ci/vps'
mkdir -p "$PREFIX"
read -r _ SHA _ # oldrev newrev refname
FILENAME="$(date -Is)-$SHA.log"
LOGFILE="$PREFIX/$FILENAME"
exec &> >(tee -a "$LOGFILE")
echo "Starting CI job at: $(date -Is)"
finish() {
STATUS="$?"
printf "\n\n>>> exit status was %s\n" "$STATUS"
echo "Finishing CI job at: $(date -Is)"
popd
NOTE=$(cat <<EOF
See CI logs with:
git notes --ref=refs/notes/ci-logs show $SHA
git notes --ref=refs/notes/ci-data show $SHA
EOF
)
git notes --ref=refs/notes/ci-data add -f -m "$STATUS $FILENAME"
git notes --ref=refs/notes/ci-logs add -f -F "$LOGFILE"
git notes add -f -m "$NOTE"
printf "\n\n>>> CI logs added as Git note."
}
trap finish EXIT
unset GIT_DIR
CLONE="$(mktemp -d)"
git clone . "$CLONE"
pushd "$CLONE"
git config --global user.email git@euandre.org
git config --global user.name 'EuAndreh CI'
./container make check site
./container make publish</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="git-notes-ci.html.0.txt">plaintext</a></p>
</div>
<div class="paragraph">
<p>The important part is in the <code>finish()</code> function: - #25 stores the exit status and the
generated filename separated by spaces; - #26 adds the log file in a note using the
<code>refs/notes/ci-logs</code> ref; - #27 it adds a note to the commit saying how to see the logs.</p>
</div>
<div class="paragraph">
<p>A commit now has an attached note, and shows it whenever you look at it:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B2-L1"
href="#B2-L1">1</a>
<a class="code-line-anchor"
id="B2-L2"
href="#B2-L2">2</a>
<a class="code-line-anchor"
id="B2-L3"
href="#B2-L3">3</a>
<a class="code-line-anchor"
id="B2-L4"
href="#B2-L4">4</a>
<a class="code-line-anchor"
id="B2-L5"
href="#B2-L5">5</a>
<a class="code-line-anchor"
id="B2-L6"
href="#B2-L6">6</a>
<a class="code-line-anchor"
id="B2-L7"
href="#B2-L7">7</a>
<a class="code-line-anchor"
id="B2-L8"
href="#B2-L8">8</a>
<a class="code-line-anchor"
id="B2-L9"
href="#B2-L9">9</a>
<a class="code-line-anchor"
id="B2-L10"
href="#B2-L10">10</a>
<a class="code-line-anchor"
id="B2-L11"
href="#B2-L11">11</a>
<a class="code-line-anchor"
id="B2-L12"
href="#B2-L12">12</a>
<a class="code-line-anchor"
id="B2-L13"
href="#B2-L13">13</a>
<a class="code-line-anchor"
id="B2-L14"
href="#B2-L14">14</a>
<a class="code-line-anchor"
id="B2-L15"
href="#B2-L15">15</a>
<a class="code-line-anchor"
id="B2-L16"
href="#B2-L16">16</a>
<a class="code-line-anchor"
id="B2-L17"
href="#B2-L17">17</a>
<a class="code-line-anchor"
id="B2-L18"
href="#B2-L18">18</a>
<a class="code-line-anchor"
id="B2-L19"
href="#B2-L19">19</a>
<a class="code-line-anchor"
id="B2-L20"
href="#B2-L20">20</a>
<a class="code-line-anchor"
id="B2-L21"
href="#B2-L21">21</a>
<a class="code-line-anchor"
id="B2-L22"
href="#B2-L22">22</a>
<a class="code-line-anchor"
id="B2-L23"
href="#B2-L23">23</a>
<a class="code-line-anchor"
id="B2-L24"
href="#B2-L24">24</a>
<a class="code-line-anchor"
id="B2-L25"
href="#B2-L25">25</a>
<a class="code-line-anchor"
id="B2-L26"
href="#B2-L26">26</a>
<a class="code-line-anchor"
id="B2-L27"
href="#B2-L27">27</a></pre>
</td>
<td>
<pre>$ git show 87c57133abd8be5d7cc46afbf107f59b26066575
commit 87c57133abd8be5d7cc46afbf107f59b26066575
Author: EuAndreh <eu@euandre.org>
Date: Wed Feb 24 21:58:28 2021 -0300
vps/machines.scm: Change path to cronjob files
Notes:
See CI logs with:
git notes --ref=refs/notes/ci-logs show 87c57133abd8be5d7cc46afbf107f59b26066575
git notes --ref=refs/notes/ci-data show 87c57133abd8be5d7cc46afbf107f59b26066575
diff --git a/servers/vps/machines.scm b/servers/vps/machines.scm
index d1830ca..a4ccde7 100644
--- a/servers/vps/machines.scm
+++ b/servers/vps/machines.scm
@@ -262,8 +262,8 @@ pki " mail-domain " key \"" (tls-priv-for mail-domain) "\""))
(service mcron-service-type
(mcron-configuration
(jobs
- (list #~(job "30 1 * * 1" "guix gc -d")
- #~(job "30 0 * * *" "/var/lib/euandreh/backup.sh")))))
+ (list #~(job "30 1 * * 1" "/opt/bin/gc.sh")
+ #~(job "30 0 * * *" "/opt/bin/backup.sh")))))
(service dhcp-client-service-type)
#;
(service opensmtpd-service-type</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="git-notes-ci.html.1.txt">plaintext</a></p>
</div>
<div class="paragraph">
<p>Other tools such as <a href="https://git.zx2c4.com/cgit/">cgit</a> will also show notes on the web
interface: <a href="https://euandre.org/git/servers/commit?id=87c57133abd8be5d7cc46afbf107f59b26066575"
class="bare">https://euandre.org/git/servers/commit?id=87c57133abd8be5d7cc46afbf107f59b26066575</a>.</p>
</div>
<div class="paragraph">
<p>You can go even further: since cgit can serve raw blob directly, you can even serve such artifacts (log
files, release artifacts, binaries) from cgit itself:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B3-L1"
href="#B3-L1">1</a>
<a class="code-line-anchor"
id="B3-L2"
href="#B3-L2">2</a>
<a class="code-line-anchor"
id="B3-L3"
href="#B3-L3">3</a></pre>
</td>
<td>
<pre>$ SHA="$(git notes --ref=refs/notes/ci-logs list 87c57133abd8be5d7cc46afbf107f59b26066575)"
$ echo "https://euandre.org/git/servers/blob?id=$SHA"
https://euandre.org/git/servers/blob?id=1707a97bae24e3864fe7943f8dda6d01c294fb5c</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="git-notes-ci.html.2.txt">plaintext</a></p>
</div>
<div class="paragraph">
<p>And like that you’ll have cgit serving the artifacts for you: <a href=
"https://euandre.org/git/servers/blob?id=1707a97bae24e3864fe7943f8dda6d01c294fb5c"
class="bare">https://euandre.org/git/servers/blob?id=1707a97bae24e3864fe7943f8dda6d01c294fb5c</a>.</p>
</div>
Git bisect automation
2020-11-12T00:00:00+00:00
https://euandre.org/en/til/2020/11/12/git-bisect-automation.html
EuAndreh
eu@euandre.org
<div class="paragraph">
<p>It is good to have an standardized way to run builds and tests on the repository of a project, so that you
can find when a bug was introduced by using <code>git bisect run</code>.</p>
</div>
<div class="paragraph">
<div class="paragraph">
<p>It is good to have an standardized way to run builds and tests on the repository of a project, so that you
can find when a bug was introduced by using <code>git bisect run</code>.</p>
</div>
<div class="paragraph">
<p>I’ve already been in the situation when a bug was introduced and I didn’t know how it even was occurring,
and running Git bisect over hundreds of commits to pinpoint the failing commit was very empowering:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B1-L1"
href="#B1-L1">1</a>
<a class="code-line-anchor"
id="B1-L2"
href="#B1-L2">2</a>
<a class="code-line-anchor"
id="B1-L3"
href="#B1-L3">3</a></pre>
</td>
<td>
<pre>$ GOOD_COMMIT_SHA=e1fd0a817d192c5a5df72dd7422e36558fa78e46
$ git bisect start HEAD $GOOD_COMMIT_SHA
$ git bisect run sn -c './build.sh && ./run-failing-case.sh'</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="git-bisect-automation.html.0.txt">plaintext</a></p>
</div>
<div class="paragraph">
<p>Git will than do a binary search between the commits, and run the commands you provide it with to find the
failing commit.</p>
</div>
<div class="paragraph">
<p>Instead of being afraid of doing a bisect, you should instead leverage it, and make Git help you dig
through the history of the repository to find the bad code.</p>
</div>
Search changes to a filename pattern in Git history
2020-10-11T00:00:00+00:00
https://euandre.org/en/til/2020/10/11/search-git-history.html
EuAndreh
eu@euandre.org
<div class="paragraph">
<p>This is <a href="../../08/14/browse-git.html">yet</a> <a href="../../08/16/git-search.html">another</a>
<a href="../../08/28/grep-online.html">"search in Git"</a> TIL entry. You could say that Git has a
unintuitive CLI, or that is it very powerful.</p>
</div>
<div class="paragraph">
<p>This is <a href="../../08/14/browse-git.html">yet</a> <a href="../../08/16/git-search.html">another</a>
<a href="../../08/28/grep-online.html">"search in Git"</a> TIL entry. You could say that Git has a
unintuitive CLI, or that is it very powerful.</p>
</div>
<div class="paragraph">
<p>I wanted to search for an old file that I new that was in the history of the repository, but was deleted
some time ago. So I didn’t really remember the name, only bits of it.</p>
</div>
<div class="paragraph">
<p>I immediately went to the list of TILs I had written on searching in Git, but it wasn’t readily obvious
how to do it, so here it goes:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B1-L1"
href="#B1-L1">1</a></pre>
</td>
<td>
<pre>git log -- *pattern*</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="search-git-history.html.0.txt">plaintext</a></p>
</div>
<div class="paragraph">
<p>You could add globs before the pattern to match things on any directory, and add our <code>-p</code>
friend to promptly see the diffs:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B2-L1"
href="#B2-L1">1</a></pre>
</td>
<td>
<pre>git log -p -- **/*pattern*</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="search-git-history.html.1.txt">plaintext</a></p>
</div>
Pull requests with Git, the old school way
2020-09-05T00:00:00+00:00
https://euandre.org/en/til/2020/09/05/oldschool-pr.html
EuAndreh
eu@euandre.org
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>It might be news to you, as it was to me, that "pull requests" that you can create on a Git hosting
provider’s web UIfootnote:pr-webui[ And maybe even using the Git hosting provider’s API from the command
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>It might be news to you, as it was to me, that "pull requests" that you can create on a Git hosting
provider’s web UIfootnote:pr-webui[ And maybe even using the Git hosting provider’s API from the command
line! ] like GitLab/Bitbucket/GitHub actually comes from Git itself: <code>git request-pull</code>.</p>
</div>
<div class="paragraph">
<p>At the very core, they accomplish the same thing: both the original and the web UI ones are ways for
you to request the project maintainers to pull in your changes from your fork. It’s like saying: "hi
there, I did some changes on my clone of the repository, what do you think about bringing those in?".</p>
</div>
<div class="paragraph">
<p>The only difference is that you’re working with only Git itself, so you’re not tied to any Git hosting
provider: you can send pull requests across them transparently! You could even use your own <a href=
"https://git.zx2c4.com/cgit/">cgit</a> installation. No need to be locked in by any of them, putting the
"D" back in "DVCS": it’s a <strong>distributed</strong> version control system.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="git-request-pull-introduction"><code>git request-pull</code> introduction<a class="anchor"
href="#git-request-pull-introduction"></a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Here’s the raw output of a <code>git request-pull</code>:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B1-L1"
href="#B1-L1">1</a>
<a class="code-line-anchor"
id="B1-L2"
href="#B1-L2">2</a>
<a class="code-line-anchor"
id="B1-L3"
href="#B1-L3">3</a>
<a class="code-line-anchor"
id="B1-L4"
href="#B1-L4">4</a>
<a class="code-line-anchor"
id="B1-L5"
href="#B1-L5">5</a>
<a class="code-line-anchor"
id="B1-L6"
href="#B1-L6">6</a>
<a class="code-line-anchor"
id="B1-L7"
href="#B1-L7">7</a>
<a class="code-line-anchor"
id="B1-L8"
href="#B1-L8">8</a>
<a class="code-line-anchor"
id="B1-L9"
href="#B1-L9">9</a>
<a class="code-line-anchor"
id="B1-L10"
href="#B1-L10">10</a>
<a class="code-line-anchor"
id="B1-L11"
href="#B1-L11">11</a>
<a class="code-line-anchor"
id="B1-L12"
href="#B1-L12">12</a>
<a class="code-line-anchor"
id="B1-L13"
href="#B1-L13">13</a>
<a class="code-line-anchor"
id="B1-L14"
href="#B1-L14">14</a>
<a class="code-line-anchor"
id="B1-L15"
href="#B1-L15">15</a>
<a class="code-line-anchor"
id="B1-L16"
href="#B1-L16">16</a>
<a class="code-line-anchor"
id="B1-L17"
href="#B1-L17">17</a>
<a class="code-line-anchor"
id="B1-L18"
href="#B1-L18">18</a>
<a class="code-line-anchor"
id="B1-L19"
href="#B1-L19">19</a></pre>
</td>
<td>
<pre>$ git request-pull HEAD public-origin
The following changes since commit 302c9f2f035c0360acd4e13142428c100a10d43f:
db post: Add link to email exchange (2020-09-03 21:23:55 -0300)
are available in the Git repository at:
https://euandre.org/git/euandre.org/
for you to fetch changes up to 524c646cdac4153e54f2163e280176adbc4873fa:
db post: better pinpoint sqlite unsuitability (2020-09-03 22:08:56 -0300)
----------------------------------------------------------------
EuAndreh (1):
db post: better pinpoint sqlite unsuitability
_posts/2020-08-31-the-database-i-wish-i-had.md | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="oldschool-pr.html.0.txt">plaintext</a></p>
</div>
<div class="paragraph">
<p>That very first line is saying: "create me a pull request with only a single commit, defined by
<code>HEAD</code>, and use the URL defined by <code>public-origin</code>".</p>
</div>
<div class="paragraph">
<p>Here’s a pitfall: you may try using your <code>origin</code> remote at first where I put
<code>public-origin</code>, but that is many times pointing to something like
<code>git@example.com</code>, or <code>git.example.com:repo.git</code> (check that with <code>git remote
-v | grep origin</code>). On both cases those are addresses available for interaction via SSH, and it
would be better if your pull requests used an address ready for public consumption.</p>
</div>
<div class="paragraph">
<p>A simple solution for that is for you to add the <code>public-origin</code> alias as the HTTPS
alternative to the SSH version:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B2-L1"
href="#B2-L1">1</a></pre>
</td>
<td>
<pre>$ git remote add public-origin https://example.com/user/repo</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="oldschool-pr.html.1.txt">plaintext</a></p>
</div>
<div class="paragraph">
<p>Every Git hosting provider exposes repositories via HTTPS.</p>
</div>
<div class="paragraph">
<p>Experiment it yourself, and get acquainted with the CLI.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="delivering-decentralized-pull-requests">Delivering decentralized pull requests<a class="anchor"
href="#delivering-decentralized-pull-requests"></a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Now that you can create the content of a pull request, you can just <a href=
"../04/cli-email-fun-profit.html">deliver it</a> to the interested parties email:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B3-L1"
href="#B3-L1">1</a>
<a class="code-line-anchor"
id="B3-L2"
href="#B3-L2">2</a>
<a class="code-line-anchor"
id="B3-L3"
href="#B3-L3">3</a>
<a class="code-line-anchor"
id="B3-L4"
href="#B3-L4">4</a>
<a class="code-line-anchor"
id="B3-L5"
href="#B3-L5">5</a>
<a class="code-line-anchor"
id="B3-L6"
href="#B3-L6">6</a>
<a class="code-line-anchor"
id="B3-L7"
href="#B3-L7">7</a>
<a class="code-line-anchor"
id="B3-L8"
href="#B3-L8">8</a>
<a class="code-line-anchor"
id="B3-L9"
href="#B3-L9">9</a>
<a class="code-line-anchor"
id="B3-L10"
href="#B3-L10">10</a>
<a class="code-line-anchor"
id="B3-L11"
href="#B3-L11">11</a></pre>
</td>
<td>
<pre># send a PR with your last commit to the author's email
git request-pull HEAD public-origin | mail author@example.com -s "PR: Add thing to repo"
# send a PR with your last 5 commits to the project's mailing
# list, including the patch
git request-pull -p HEAD~5 public-origin | \
mail list@example.com -s "PR: Add another thing to repo"
# send every commit that is new in "other-branch"
git request-pull master public-origin other-branch | \
mail list@example.com -s 'PR: All commits from my "other-brach"'</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="oldschool-pr.html.2.txt">plaintext</a></p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="conclusion">Conclusion<a class="anchor"
href="#conclusion"></a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>In practice, I’ve never used or seen anyone use pull requests this way: everybody is just <a href=
"https://drewdevault.com/2018/07/23/Git-is-already-distributed.html">sending patches via email</a>.</p>
</div>
<div class="paragraph">
<p>If you stop to think about this model, the problem of "Git hosting providers becoming too centralized"
is a non-issue, and "Git federation" proposals are a less attractive as they may sound initially.</p>
</div>
<div class="paragraph">
<p>Using Git this way is not scary or so weird as the first impression may suggest. It is actually how
Git was designed to be used.</p>
</div>
<div class="paragraph">
<p>Check <code>git help request-pull</code> for more info.</p>
</div>
</div>
</div>
Grep online repositories
2020-08-28T00:00:00+00:00
https://euandre.org/en/til/2020/08/28/grep-online.html
EuAndreh
eu@euandre.org
<div class="paragraph">
<p>I often find interesting source code repositories online that I want to grep for some pattern but I can’t,
because either:</p>
</div>
<div class="ulist">
<div class="paragraph">
<p>I often find interesting source code repositories online that I want to grep for some pattern but I can’t,
because either:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>the repository is on <a href="https://git.zx2c4.com/cgit/">cgit</a> or a similar code repository that
doesn’t allow search in files, or;</p>
</li>
<li>
<p>the search function is really bad, and doesn’t allow me to use regular expressions for searching
patterns in the code.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Here’s a simple script that allows you to overcome that problem easily:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B1-L1"
href="#B1-L1">1</a>
<a class="code-line-anchor"
id="B1-L2"
href="#B1-L2">2</a>
<a class="code-line-anchor"
id="B1-L3"
href="#B1-L3">3</a>
<a class="code-line-anchor"
id="B1-L4"
href="#B1-L4">4</a>
<a class="code-line-anchor"
id="B1-L5"
href="#B1-L5">5</a>
<a class="code-line-anchor"
id="B1-L6"
href="#B1-L6">6</a>
<a class="code-line-anchor"
id="B1-L7"
href="#B1-L7">7</a>
<a class="code-line-anchor"
id="B1-L8"
href="#B1-L8">8</a>
<a class="code-line-anchor"
id="B1-L9"
href="#B1-L9">9</a>
<a class="code-line-anchor"
id="B1-L10"
href="#B1-L10">10</a>
<a class="code-line-anchor"
id="B1-L11"
href="#B1-L11">11</a>
<a class="code-line-anchor"
id="B1-L12"
href="#B1-L12">12</a>
<a class="code-line-anchor"
id="B1-L13"
href="#B1-L13">13</a>
<a class="code-line-anchor"
id="B1-L14"
href="#B1-L14">14</a>
<a class="code-line-anchor"
id="B1-L15"
href="#B1-L15">15</a>
<a class="code-line-anchor"
id="B1-L16"
href="#B1-L16">16</a>
<a class="code-line-anchor"
id="B1-L17"
href="#B1-L17">17</a>
<a class="code-line-anchor"
id="B1-L18"
href="#B1-L18">18</a>
<a class="code-line-anchor"
id="B1-L19"
href="#B1-L19">19</a>
<a class="code-line-anchor"
id="B1-L20"
href="#B1-L20">20</a>
<a class="code-line-anchor"
id="B1-L21"
href="#B1-L21">21</a>
<a class="code-line-anchor"
id="B1-L22"
href="#B1-L22">22</a>
<a class="code-line-anchor"
id="B1-L23"
href="#B1-L23">23</a>
<a class="code-line-anchor"
id="B1-L24"
href="#B1-L24">24</a>
<a class="code-line-anchor"
id="B1-L25"
href="#B1-L25">25</a>
<a class="code-line-anchor"
id="B1-L26"
href="#B1-L26">26</a>
<a class="code-line-anchor"
id="B1-L27"
href="#B1-L27">27</a>
<a class="code-line-anchor"
id="B1-L28"
href="#B1-L28">28</a>
<a class="code-line-anchor"
id="B1-L29"
href="#B1-L29">29</a>
<a class="code-line-anchor"
id="B1-L30"
href="#B1-L30">30</a>
<a class="code-line-anchor"
id="B1-L31"
href="#B1-L31">31</a>
<a class="code-line-anchor"
id="B1-L32"
href="#B1-L32">32</a>
<a class="code-line-anchor"
id="B1-L33"
href="#B1-L33">33</a>
<a class="code-line-anchor"
id="B1-L34"
href="#B1-L34">34</a>
<a class="code-line-anchor"
id="B1-L35"
href="#B1-L35">35</a>
<a class="code-line-anchor"
id="B1-L36"
href="#B1-L36">36</a>
<a class="code-line-anchor"
id="B1-L37"
href="#B1-L37">37</a>
<a class="code-line-anchor"
id="B1-L38"
href="#B1-L38">38</a>
<a class="code-line-anchor"
id="B1-L39"
href="#B1-L39">39</a>
<a class="code-line-anchor"
id="B1-L40"
href="#B1-L40">40</a></pre>
</td>
<td>
<pre>#!/usr/bin/env bash
set -eu
end="\033[0m"
red="\033[0;31m"
red() { echo -e "${red}${1}${end}"; }
usage() {
red "Missing argument $1.\n"
cat <<EOF
Usage:
$0 <REGEX_PATTERN> <REPOSITORY_URL>
Arguments:
REGEX_PATTERN Regular expression that "git grep" can search
REPOSITORY_URL URL address that "git clone" can download the repository from
Examples:
Searching "make get-git" in cgit repository:
git search 'make get-git' https://git.zx2c4.com/cgit/
git search 'make get-git' https://git.zx2c4.com/cgit/ -- \$(git rev-list --all)
EOF
exit 2
}
REGEX_PATTERN="${1:-}"
REPOSITORY_URL="${2:-}"
[[ -z "${REGEX_PATTERN}" ]] && usage 'REGEX_PATTERN'
[[ -z "${REPOSITORY_URL}" ]] && usage 'REPOSITORY_URL'
mkdir -p /tmp/git-search
DIRNAME="$(echo "${REPOSITORY_URL%/}" | rev | cut -d/ -f1 | rev)"
if [[ ! -d "/tmp/git-search/${DIRNAME}" ]]; then
git clone "${REPOSITORY_URL}" "/tmp/git-search/${DIRNAME}"
fi
pushd "/tmp/git-search/${DIRNAME}"
shift 3 || shift 2 # when "--" is missing
git grep "${REGEX_PATTERN}" "${@}"</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="grep-online.html.0.txt">plaintext</a></p>
</div>
<div class="paragraph">
<p>It is a wrapper around <code>git grep</code> that downloads the repository when missing. Save in a file
called <code>git-search</code>, make the file executable and add it to your path.</p>
</div>
<div class="paragraph">
<p>Overview:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><em>lines 1~2</em>:</p>
<div class="paragraph">
<p>Bash shebang and the <code>set -eu</code> options to exit on error or undefined variables.</p>
</div>
</li>
<li>
<p><em>lines 4~30</em>:</p>
<div class="paragraph">
<p>Usage text to be printed when providing less arguments than expected.</p>
</div>
</li>
<li>
<p><em>line 33</em>:</p>
<div class="paragraph">
<p>Extract the repository name from the URL, removing trailing slashes.</p>
</div>
</li>
<li>
<p><em>lines 34~37</em>:</p>
<div class="paragraph">
<p>Download the repository when missing and go to the folder.</p>
</div>
</li>
<li>
<p><em>line 39</em>:</p>
<div class="paragraph">
<p>Make the variable <code>$@</code> contain the rest of the unused arguments.</p>
</div>
</li>
<li>
<p><em>line 40</em>:</p>
<div class="paragraph">
<p>Perform <code>git grep</code>, forwarding the remaining arguments from <code>$@</code>.</p>
</div>
</li>
</ul>
</div>
<div class="paragraph">
<p>Example output:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B2-L1"
href="#B2-L1">1</a>
<a class="code-line-anchor"
id="B2-L2"
href="#B2-L2">2</a>
<a class="code-line-anchor"
id="B2-L3"
href="#B2-L3">3</a>
<a class="code-line-anchor"
id="B2-L4"
href="#B2-L4">4</a>
<a class="code-line-anchor"
id="B2-L5"
href="#B2-L5">5</a>
<a class="code-line-anchor"
id="B2-L6"
href="#B2-L6">6</a>
<a class="code-line-anchor"
id="B2-L7"
href="#B2-L7">7</a>
<a class="code-line-anchor"
id="B2-L8"
href="#B2-L8">8</a>
<a class="code-line-anchor"
id="B2-L9"
href="#B2-L9">9</a>
<a class="code-line-anchor"
id="B2-L10"
href="#B2-L10">10</a>
<a class="code-line-anchor"
id="B2-L11"
href="#B2-L11">11</a>
<a class="code-line-anchor"
id="B2-L12"
href="#B2-L12">12</a>
<a class="code-line-anchor"
id="B2-L13"
href="#B2-L13">13</a>
<a class="code-line-anchor"
id="B2-L14"
href="#B2-L14">14</a>
<a class="code-line-anchor"
id="B2-L15"
href="#B2-L15">15</a></pre>
</td>
<td>
<pre>$ git search 'make get-git' https://git.zx2c4.com/cgit/
Clonage dans '/tmp/git-search/cgit'...
remote: Enumerating objects: 542, done.
remote: Counting objects: 100% (542/542), done.
remote: Compressing objects: 100% (101/101), done.
warning: object 51dd1eff1edc663674df9ab85d2786a40f7ae3a5: gitmodulesParse: could not parse gitmodules blob
remote: Total 7063 (delta 496), reused 446 (delta 441), pack-reused 6521
Réception d'objets: 100% (7063/7063), 8.69 Mio | 5.39 Mio/s, fait.
Résolution des deltas: 100% (5047/5047), fait.
/tmp/git-search/cgit ~/dev/libre/songbooks/docs
README: $ make get-git
$ git search 'make get-git' https://git.zx2c4.com/cgit/
/tmp/git-search/cgit ~/dev/libre/songbooks/docs
README: $ make get-git</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="grep-online.html.1.txt">plaintext</a></p>
</div>
<div class="paragraph">
<p>Subsequent greps on the same repository are faster because no download is needed.</p>
</div>
<div class="paragraph">
<p>When no argument is provided, it prints the usage text:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B3-L1"
href="#B3-L1">1</a>
<a class="code-line-anchor"
id="B3-L2"
href="#B3-L2">2</a>
<a class="code-line-anchor"
id="B3-L3"
href="#B3-L3">3</a>
<a class="code-line-anchor"
id="B3-L4"
href="#B3-L4">4</a>
<a class="code-line-anchor"
id="B3-L5"
href="#B3-L5">5</a>
<a class="code-line-anchor"
id="B3-L6"
href="#B3-L6">6</a>
<a class="code-line-anchor"
id="B3-L7"
href="#B3-L7">7</a>
<a class="code-line-anchor"
id="B3-L8"
href="#B3-L8">8</a>
<a class="code-line-anchor"
id="B3-L9"
href="#B3-L9">9</a>
<a class="code-line-anchor"
id="B3-L10"
href="#B3-L10">10</a>
<a class="code-line-anchor"
id="B3-L11"
href="#B3-L11">11</a>
<a class="code-line-anchor"
id="B3-L12"
href="#B3-L12">12</a>
<a class="code-line-anchor"
id="B3-L13"
href="#B3-L13">13</a>
<a class="code-line-anchor"
id="B3-L14"
href="#B3-L14">14</a></pre>
</td>
<td>
<pre>$ git search
Missing argument REGEX_PATTERN.
Usage:
/home/andreh/dev/libre/dotfiles/scripts/ad-hoc/git-search <REGEX_PATTERN> <REPOSITORY_URL>
Arguments:
REGEX_PATTERN Regular expression that "git grep" can search
REPOSITORY_URL URL address that "git clone" can download the repository from
Examples:
Searching "make get-git" in cgit repository:
git search 'make get-git' https://git.zx2c4.com/cgit/
git search 'make get-git' https://git.zx2c4.com/cgit/ -- $(git rev-list --all)</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="grep-online.html.2.txt">plaintext</a></p>
</div>
Search in git
2020-08-16T00:00:00+00:00
https://euandre.org/en/til/2020/08/16/git-search.html
EuAndreh
eu@euandre.org
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>Here’s a useful trio to know about to help you search things in git:</p>
</div>
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>Here’s a useful trio to know about to help you search things in git:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p><code>git show <commit></code></p>
</li>
<li>
<p><code>git log --grep='<regexp>'</code></p>
</li>
<li>
<p><code>git grep '<regexp>' [commit]</code></p>
</li>
</ol>
</div>
</div>
</div>
<div class="sect1">
<h2 id="1-git-show-commit">1. <code>git show <commit></code><a class="anchor"
href="#1-git-show-commit"></a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Show a specific commit and it’s diff:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B1-L1"
href="#B1-L1">1</a>
<a class="code-line-anchor"
id="B1-L2"
href="#B1-L2">2</a>
<a class="code-line-anchor"
id="B1-L3"
href="#B1-L3">3</a>
<a class="code-line-anchor"
id="B1-L4"
href="#B1-L4">4</a>
<a class="code-line-anchor"
id="B1-L5"
href="#B1-L5">5</a>
<a class="code-line-anchor"
id="B1-L6"
href="#B1-L6">6</a></pre>
</td>
<td>
<pre>git show
# shows the latest commit
git show <commit>
# shows an specific <commit>
git show v1.2
# shows commit tagged with v1.2</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="git-search.html.0.txt">plaintext</a></p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="2-git-log-grepregexp">2. <code>git log --grep='<regexp>'</code><a class="anchor"
href="#2-git-log-grepregexp"></a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Search through the commit messages:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B2-L1"
href="#B2-L1">1</a></pre>
</td>
<td>
<pre>git log --grep='refactor'</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="git-search.html.1.txt">plaintext</a></p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="3-git-grep-regexp-commit">3. <code>git grep '<regexp>' [commit]</code><a class="anchor"
href="#3-git-grep-regexp-commit"></a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Search content in git history:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B3-L1"
href="#B3-L1">1</a>
<a class="code-line-anchor"
id="B3-L2"
href="#B3-L2">2</a>
<a class="code-line-anchor"
id="B3-L3"
href="#B3-L3">3</a>
<a class="code-line-anchor"
id="B3-L4"
href="#B3-L4">4</a></pre>
</td>
<td>
<pre>git grep 'TODO'
# search the repository for the "TODO" string
git grep 'TODO' $(git rev-list --all)
# search the whole history for "TODO" string</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="git-search.html.2.txt">plaintext</a></p>
</div>
<div class="paragraph">
<p>And if you find an occurrence of the regexp in a specific commit and you want to browse the repository
in that point in time, you can link:../14/browse-git.html[use git checkout for that].</p>
</div>
</div>
</div>
Browse a git repository at a specific commit
2020-08-14T00:00:00+00:00
https://euandre.org/en/til/2020/08/14/browse-git.html
EuAndreh
eu@euandre.org
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>I commonly use tools like <code>git log</code> together with <code>git show</code> when inspecting
past changes in a repository:</p>
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>I commonly use tools like <code>git log</code> together with <code>git show</code> when inspecting
past changes in a repository:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B1-L1"
href="#B1-L1">1</a>
<a class="code-line-anchor"
id="B1-L2"
href="#B1-L2">2</a>
<a class="code-line-anchor"
id="B1-L3"
href="#B1-L3">3</a>
<a class="code-line-anchor"
id="B1-L4"
href="#B1-L4">4</a></pre>
</td>
<td>
<pre>git log
# search for a the commit I'm looking for
git show <my-commit>
# see the diff for the commit</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="browse-git.html.0.txt">plaintext</a></p>
</div>
<div class="paragraph">
<p>But I also wanted to not only be able to look at the diff of a specific commit, but to browse the
whole repository at that specific commit.</p>
</div>
<div class="paragraph">
<p>I used to accomplish it the "brute force" way: clone the whole repository in another folder and
checkout the commit there:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B2-L1"
href="#B2-L1">1</a>
<a class="code-line-anchor"
id="B2-L2"
href="#B2-L2">2</a>
<a class="code-line-anchor"
id="B2-L3"
href="#B2-L3">3</a></pre>
</td>
<td>
<pre>git clone <original-repo> /tmp/tmp-repo-clone
cd /tmp-repo-clone
git checkout <my-commit></pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="browse-git.html.1.txt">plaintext</a></p>
</div>
<div class="paragraph">
<p>But git itself allows we to specific the directory of the checkout by using the
<code>--work-tree</code> global git flag. This is what <code>man git</code> says about it:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B3-L1"
href="#B3-L1">1</a>
<a class="code-line-anchor"
id="B3-L2"
href="#B3-L2">2</a>
<a class="code-line-anchor"
id="B3-L3"
href="#B3-L3">3</a>
<a class="code-line-anchor"
id="B3-L4"
href="#B3-L4">4</a>
<a class="code-line-anchor"
id="B3-L5"
href="#B3-L5">5</a></pre>
</td>
<td>
<pre>--work-tree=<path>
Set the path to the working tree. It can be an absolute path or a path relative to the current working
directory. This can also be controlled by setting the GIT_WORK_TREE environment variable and the
core.worktree configuration variable (see core.worktree in git-config(1) for a more detailed
discussion).</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="browse-git.html.2.txt">plaintext</a></p>
</div>
<div class="paragraph">
<p>So it allows us to set the desired path of the working tree. So if we want to copy the contents of the
current working tree into <code>copy/</code>:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B4-L1"
href="#B4-L1">1</a>
<a class="code-line-anchor"
id="B4-L2"
href="#B4-L2">2</a></pre>
</td>
<td>
<pre>mkdir copy
git --work-tree=copy/ checkout .</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="browse-git.html.3.txt">plaintext</a></p>
</div>
<div class="paragraph">
<p>After that <code>copy/</code> will contain a replica of the code in HEAD. But to checkout a specific,
we need some extra parameters:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B5-L1"
href="#B5-L1">1</a></pre>
</td>
<td>
<pre>git --work-tree=<dir> checkout <my-commit> -- .</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="browse-git.html.4.txt">plaintext</a></p>
</div>
<div class="paragraph">
<p>There’s an extra <code>– .</code> at the end, which initially looks like we’re sending Morse signals
to git, but we’re actually saying to <code>git-checkout</code> which sub directory of
<code><my-commit></code> we want to look at. Which means we can do something like:</p>
</div>
<div class="listingblock">
<div class="content">
<table>
<tbody>
<tr>
<td>
<pre class="lineno"><a class="code-line-anchor"
id="B6-L1"
href="#B6-L1">1</a></pre>
</td>
<td>
<pre>git --work-tree=<dir> checkout <my-commit> -- src/</pre>
</td>
</tr>
</tbody>
</table>
</div>
</div>
<div class="paragraph plaintext">
<p><a href="browse-git.html.5.txt">plaintext</a></p>
</div>
<div class="paragraph">
<p>And with that <code><dir></code> will only contain what was inside <code>src/</code> at
<code><commit></code>.</p>
</div>
<div class="paragraph">
<p>After any of those checkouts, you have to <code>git reset .</code> to reset your current staging area
back to what it was before the checkout.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="references">References<a class="anchor"
href="#references"></a></h2>
<div class="sectionbody">
<div class="olist arabic">
<ol class="arabic">
<li>
<p><a href="https://stackoverflow.com/a/16493707">GIT: Checkout to a specific folder</a>
(StackOverflow)</p>
</li>
</ol>
</div>
</div>
</div>