Hi,

I am having troubles keeping all the group's repositories updated.
I double-checked with git.html and everything looks as it should.

The problem seems to be that 'mr up' skips repositories it should not
skip. When I run 'gbp pull' in them, there are new commits fetched.

Example:

$ ./compare-lastactivity libgd-text-perl --debug
D: Working on package 'libgd-text-perl' ...
D: Last activity at salsa: 2021-01-22T20:23:53Z
D: Last fetch in repo: 2021-11-04T13:42:45Z

$ stat .git/FETCH_HEAD
File: .git/FETCH_HEAD
Size: 2055 Blocks: 8 IO Block: 4096 regular file Device: 2eh/46d Inode: 1037725 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 1000/ dam) Gid: ( 1000/ dam) Access: 2016-11-09 20:22:16.102958000 +0200
Modify: 2019-09-15 14:56:51.696968093 +0300
Change: 2021-11-04 15:42:45.072112725 +0200
Birth: 2021-11-04 15:42:45.072112725 +0200

$ gbp pull
gbp:info: Fetching from default remote for each branch
gbp:info: Updating 'master': c15d9f4d1561..812d0f55dead
gbp:info: Branch 'pristine-tar' is already up to date.
gbp:info: Branch 'upstream' is already up to date.

compare-lastactivity looks at the ctime (i-node/metadata change time).
Perhaps it should look at the mtime (data change time)?

The filesystem where the repositories reside is mounted with the
'relatime' option and perhaps this has some side effects not
considered by compare-lastactivity?

Thanks for any insights,
Damyan

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sun, 26 Jun 2022 23:37:42 +0300, Damyan Ivanov wrote:

$ stat .git/FETCH_HEAD
File: .git/FETCH_HEAD
Size: 2055 Blocks: 8 IO Block: 4096 regular file Device: 2eh/46d Inode: 1037725 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 1000/ dam) Gid: ( 1000/ dam) Access: 2016-11-09 20:22:16.102958000 +0200
Modify: 2019-09-15 14:56:51.696968093 +0300
Change: 2021-11-04 15:42:45.072112725 +0200
Birth: 2021-11-04 15:42:45.072112725 +0200

% stat .git/FETCH_HEAD
File: .git/FETCH_HEAD
Size: 2481 Blocks: 8 IO Block: 4096 regular file Device: fe03h/65027d Inode: 7291655 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 1000/ gregoa) Gid: ( 1000/ gregoa) Access: 2021-01-31 08:30:28.390935982 +0100
Modify: 2021-01-30 20:11:20.295060888 +0100
Change: 2021-01-30 20:11:20.295060888 +0100
Birth: 2021-01-30 00:06:18.427682314 +0100

compare-lastactivity looks at the ctime (i-node/metadata change time). Perhaps it should look at the mtime (data change time)?

That's an interesting idea. Maybe worth trying? :)

The filesystem where the repositories reside is mounted with the
'relatime' option and perhaps this has some side effects not
considered by compare-lastactivity?

I don't think so, as I'm also using relatime.


Cheers,
gregor

--
.''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org
: :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06
`. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
`-

-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEE0eExbpOnYKgQTYX6uzpoAYZJqgYFAmK400NfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEQx RTEzMTZFOTNBNzYwQTgxMDREODVGQUJCM0E2ODAxODY0OUFBMDYACgkQuzpoAYZJ qgbJyQ/9GTh430u2bKYVHmVMSA/331vf3L+oiDhKzkpis9fT5wADR12CrYkLisF3 wR4HR2Ec78RMhF+fV2RasGRSaOkPSKZQKMfN7buw6OUPy0CDM23cIPcJjXrDVbgJ +H1a3KHAII1ecTNHaBSkTdpM5rWBRz3wT332yknCISKUmFQ+rBUeNNUJbWllJBPh aDrR8RQtmVuh1IUQDZ72vzC4LLbWTHfSpkdCBPkAeyrTNsfsRbQAog4JsaOk5qrb wSBr1RjLRKODO9BLM/4a6EMGyx/jiqWWOLlgIdV1TmCYio+M6UmryFkDzLx/r3a7 gn3ofjzfQo3kHZOYfY0hCpRPYghQjPYsNiNweXQC8ZC191Xa2iRfzGO87awJcrNN 4bOMzL6QxI5QkrQonC9j0Hy3VAMSwHnu8hw1tCtLbMW0Sv1bZlhZ3RsKXBJ0DT4k t2utCHG8EcjuBDKqUo2S0QQw2KfPpzCOg7jmjWEgEwBfTq66foN4kGvcMhbD3XSA OQqIAZmq9AQyIKht4xop1Jb4tsAAhJ+wx2QN4oKIC8nGo4ycLxkd04YqbVvHRSLm 21pLX9c7jGVhgkpjt56vPwvwg1gVA5jGn5vwjeNEeZNEpJNixNNE5IL5/CeVK9Jf 8Hv2I4QR2tHFb9VwTgfEfGfLO9qsPNeV0whWpCKJ+X487wQ12Yo=
=vsDY
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

-=| gregor herrmann, 26.06.2022 23:44:28 +0200 |=-

On Sun, 26 Jun 2022 23:37:42 +0300, Damyan Ivanov wrote:

$ stat .git/FETCH_HEAD
File: .git/FETCH_HEAD
Size: 2055 Blocks: 8 IO Block: 4096 regular file Device: 2eh/46d Inode: 1037725 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 1000/ dam) Gid: ( 1000/ dam) Access: 2016-11-09 20:22:16.102958000 +0200
Modify: 2019-09-15 14:56:51.696968093 +0300
Change: 2021-11-04 15:42:45.072112725 +0200
Birth: 2021-11-04 15:42:45.072112725 +0200

% stat .git/FETCH_HEAD
File: .git/FETCH_HEAD
Size: 2481 Blocks: 8 IO Block: 4096 regular file Device: fe03h/65027d Inode: 7291655 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 1000/ gregoa) Gid: ( 1000/ gregoa) Access: 2021-01-31 08:30:28.390935982 +0100
Modify: 2021-01-30 20:11:20.295060888 +0100
Change: 2021-01-30 20:11:20.295060888 +0100
Birth: 2021-01-30 00:06:18.427682314 +0100

compare-lastactivity looks at the ctime (i-node/metadata change time). Perhaps it should look at the mtime (data change time)?

That's an interesting idea. Maybe worth trying? :)

Tried that and it doesn't solve the problem. I think I discovered how
this happened.

When there are local changes that aren't committed, gbp pull (run by
mr up) will fetch, updating .git/FETCH_HEAD, but the merge will fail.
This leaves the repository in an old state, behind salsa.

What did help was to run 'gbp pull' over all repositories and note the
failing ones, fix them by hand or just remove them. Later 'mr up' does
the right thing. Took a lot of time, but at least now I can grep for
missing autopkgtests :)


-- Damyan

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Mon, 27 Jun 2022 22:19:19 +0300, Damyan Ivanov wrote:

compare-lastactivity looks at the ctime (i-node/metadata change time). Perhaps it should look at the mtime (data change time)?

That's an interesting idea. Maybe worth trying? :)

Tried that and it doesn't solve the problem.

Too bad but thanks for trying!

I think I discovered how
this happened.

When there are local changes that aren't committed, gbp pull (run by
mr up) will fetch, updating .git/FETCH_HEAD, but the merge will fail.
This leaves the repository in an old state, behind salsa.

Ah, I see.

Maybe there's a better file in .git than .git/FETCH_HEAD to check?

What did help was to run 'gbp pull' over all repositories and note the failing ones, fix them by hand or just remove them. Later 'mr up' does
the right thing. Took a lot of time, but at least now I can grep for
missing autopkgtests :)

So at least some success :)


Cheers,
gregor

--
.''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org
: :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06
`. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
`-

-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEE0eExbpOnYKgQTYX6uzpoAYZJqgYFAmK6F/5fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEQx RTEzMTZFOTNBNzYwQTgxMDREODVGQUJCM0E2ODAxODY0OUFBMDYACgkQuzpoAYZJ qgZPig//TBB+qRc4mArOvx9xI2nwNfdQUUCUfeIcWn1+mGDeskEcI6mOlMjeBy24 RrKj76BnEqiUTRqZUbIvDDTIS6ida3aLiCQXCY9xBAdxeZYDdstKBJ+9zAxiwQ85 DzK9Ya0NR4p3QY8clWm/0rFNTTcsKLsZZRIV40hq8f6qp+hIWfzpETjDTSBKoFfk vwGEZmgu5MKMWpzJmxrzAcw0P4ROniDZ+nk5Za1jcUcM+kdMQ3gGq9LqgTAbuywQ Id73s1l0LMv/epHVlh5wnpdbEMGEC4Ss5ryjQjbtzItQ9sOmbHav8dL6g6WHuUvA giTKXIEO4hb/VLv2HO3srezzRSGCv+/CqfqHVxiWlx56/Y5RrxTdoi63z/Mww/JA p78lDKp9NWhx0KoYuwXTiozt2l2zYeCss0u1W8TNfB8KjXVK6HRuL2rlcRht8g0T xdvRfHnpBI4/hjMAfFM25qEb2batxf+wzEBjHRH15Y4LcI2zKeyQz+2fGF+Hoxfe wgNOy89Q/Nq4ywKMzLoGZXVzx9Qp9UHosisGVfkV3lxfYyP8M1EdOdWJFRpPhzF8 V39iChlBlJDu05QCsL50ONi+R/iwO0l4ndLK8KhdFW/JmaCYXNnFao0jl/v7J3iJ XLTLBnw2VTfyRbAs93J2ufp9t76x5fBD5TylgDeLdpLQQhUT274=
=F1ri
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Tue, 28 Jun 2022 00:04:17 +0000, Ed . wrote:

It sounds to me like the command should first (before any fetching
or other activity) go through all the checkout directories, and
bomb out (with clear error messages) if any have modified files.

This morning I was thinking about your mail and about a reply
explaining that your thought is obviously correct but not trivial in
this maze of mr, dpt-salsa and compare-lastactivity … and then in the afternoon I started with an experiment in dpt-salsa which looks
promising (not pushed yet, needs another look tomorrow) -- so thanks
:)

Cheers,
gregor

--
.''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org
: :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06
`. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
`-

-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEE0eExbpOnYKgQTYX6uzpoAYZJqgYFAmK8zGxfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEQx RTEzMTZFOTNBNzYwQTgxMDREODVGQUJCM0E2ODAxODY0OUFBMDYACgkQuzpoAYZJ qgZWkQ/9E1tfSXTAUiKUeFplVUH9/z0PoWcJqBuCXaehKVN00wptCzxBhdoDEwSh DGT08P13crH7iuxA9/XKV+RI4WWmNaIKNLeud+r+iJhRc/XCEuvMUQ/QDwP+73MI 1+SJSM/YM//8iLHv1X3vJNJqsMTbfrMC6D5VipqS9yvT4/hfVEccLxQ21QLKKhAz eJmSiB3kdem8eQeI5N0Ee5/oFIegu6FPafiXia68ESSvSIGptT0PsTPz/+rjjIpI lNhVSkzzQiCdqGRiII5R9N/dMqYExe9m1YHxkn8VTuzS5rUlMn8Q9qQOy3WQJxGq njNtqoR6jtmUq37CCVPNAnLkU6I0CQc6D4HYUfE2nnqWiD48J9j6TqlbPw0G/g7x 24l6svvLv7RkQLW3CJspX0SYpn7qAeCoJAdT/st6H+cla+WD+XdnELCMrpAvq+3O HCe+v+ArZSOgQvE/buk3/3Xpl3fAnJrhoccNVuOznUo15OgIFTSRUJt+NNUhwZiE
EZIrA8p2

On Mon, 27 Jun 2022 22:19:19 +0300, Damyan Ivanov wrote:

Tried that and it doesn't solve the problem. I think I discovered how
this happened.
When there are local changes that aren't committed, gbp pull (run by
mr up) will fetch, updating .git/FETCH_HEAD, but the merge will fail.
This leaves the repository in an old state, behind salsa.

As a side note: How are you running mr?
While playing with dpt-salsa and running mr to check affects, I
noticed that I always end up in a shell -- because, as I noiticed
later -- my shell history leads me to run `mr -i -j8 up', and the -i
tells me that there is a problem.

(Of course we should improve the "experience" for plain mr without
-i, I just wanted to mention this detail.)

Cheers,
gregor

--
.''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org
: :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06
`. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
`-

-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEE0eExbpOnYKgQTYX6uzpoAYZJqgYFAmK+LXVfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEQx RTEzMTZFOTNBNzYwQTgxMDREODVGQUJCM0E2ODAxODY0OUFBMDYACgkQuzpoAYZJ qgaTGA//dh+EH5r9GXYtPA+dUwgKvbkymSupF1cATldMMQtlXqCEBBdJ3d5b7RR3 oBa1zyFG7kjmW9Ze1D5S4jugYuVQyNgSOfSS6i0e/cnNAN41+R+XbCt9A7S+pJYJ P2dr8t6lHbpnH/0NFvhGEU22fLRXTI5Hgi4rRskgl5cXXP1UfTdOwyy8rCPVYzf1 KQgpGg9pbFOd5I2wXCc+b70sdeHajU5/3mQycvhWqXo0Kme9wJ3X54fyvfCPUZS3 CASD82aXyEcU/kpH+ROT6ooDJ+Y50MTRgdzn9Kvt877ZcvWM2K6K58vchiHq5pIO j1bTyBZQbUZfeF7DQhcyw+kbhbxgzngEtfB9QUO+h3i8IYy0v/l+2d31CfXa0H4i vwA488LtunyqZwPwY7ZvuLU4EFMmEzPFxyF52MPrZLzPvpmZDBJkOABkmH8FWCva uGJ4IzIWbPCOf1Jyx4BEhJMBFChyG7N7Cq2CWcf8u9wLsAWNsgPqGBrXdHjnfwYQ 8QzcOPS4n9qaYSDotkEWGwInEz2smMcG71Kea5f6Vw374TvwNgyVT+rsa1w7hjuI xhswvnaN535vKdisIKr8qeT2bVAFa2zYE9TRSvbnv1CVvxm2YEl+6gmtMmi7BkQe nSrsUuEPt2pejUy3vUzfz2jsm+BuCXYouSowJ9th0nzrDMUbznY=
=L7BC
-----END PGP SIGNATURE-----

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

-=| gregor herrmann, 01.07.2022 01:10:43 +0200 |=-

On Mon, 27 Jun 2022 22:19:19 +0300, Damyan Ivanov wrote:

Tried that and it doesn't solve the problem. I think I discovered how
this happened.
When there are local changes that aren't committed, gbp pull (run by
mr up) will fetch, updating .git/FETCH_HEAD, but the merge will fail.
This leaves the repository in an old state, behind salsa.

As a side note: How are you running mr?

mr -j6 up

and then looking to another terminal while it runs.

And more often that not 'mr up -j6' which starts to when individual repositories get pulled :)

While playing with dpt-salsa and running mr to check affects, I
noticed that I always end up in a shell -- because, as I noiticed
later -- my shell history leads me to run `mr -i -j8 up', and the -i
tells me that there is a problem.

Oh, thanks! I didn't know about '-i'. Adding it spawns a shell in any
failing repository so that any issues can be fixed right away. Very
handy and helps avoiding the original issue.


-- Damyan

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Thu, 30 Jun 2022 00:04:25 +0200, gregor herrmann wrote:

and then in the
afternoon I started with an experiment in dpt-salsa which looks
promising (not pushed yet, needs another look tomorrow) -- so thanks
:)

Alright, I just pushed a change in dpt-salsa's mrconfig subcommand,
which is used by our .mrconfig: https://salsa.debian.org/perl-team/modules/meta/-/blob/master/.mrconfig
(at the bottom)

It now checks for dirty repos, and currently collects the information
and warns about it. (Could by changed into a failure but I'm not sure
about it …)

The runtime of `dpt salsa mrconfig -j 20' is increased (on my
laptop) from ~3 minutes to ~4 minutes. Seems ok-ish to me but maybe
there's room for some speed improvement?

Please have a look and play around :)


Cheers,
gregor

--
.''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org
: :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06
`. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
`-

-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEE0eExbpOnYKgQTYX6uzpoAYZJqgYFAmK/LO5fFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEQx RTEzMTZFOTNBNzYwQTgxMDREODVGQUJCM0E2ODAxODY0OUFBMDYACgkQuzpoAYZJ qgYJeRAAl5sBXMIsYzHngMLQc5jjNpckkUb6I9Ot3BSmbu47PhifGpyDv4OzKP4G NVIKqxLzKnpd0WgeApPF+DXVHGH9z97CkBIAMWbeST8Pdh2tN4ERxbFeiwguxenv Qc6EAkxkbauR1tvaUST+bYjj9DaeaGuIBp+wiFW5osb4u4tCyPKybAq3Spc6gRGb B4mic++2uvezpCHf+VxXvlyojUeRHQeXuHVsox+gJal5Ib/6sCSgGOyR9RlUIS99 T7qB8RKoESW2iVZJCBuOoBw2KYyDUrb5K+Z53ktrSd5mEaMTT1AhZX+jOv2wzpFu Hg5FIxobXGub6hH/P3Tj2xle++K2LXeaLMcGiUTNFDj5gl+eBU4Y9XvnT+k7Nphh QVxlMkLNy92sxqKch1S1KvyKV+L6VL9PdjrHtoqpjgQ5/hexU5iBnI0Xvln137Q8 UhHbKmnkSK7vbrEaTk+tvCgJIXS827JysfZKXg3CV0UN2pXsk3LHcn2pVL312/JG
kCcvfoFN