Hi all,

I was thinking about a report I did for my older server, which used the tradspool storage method, and that was to count how many articles are
in a group and list how many new articles have been added since last
run.

I can't see a way to make this happen using cnfs without pulling the
storage token using a similar method as 6.4 in the FAQ and then
manually parsing the newsgroups line from each article.

Do we have any better tools for interacting with cnfs...or if not,
should we?

A similar thought that when looking for text, I could grep through a
group or set of groups. Not sure this would even be possible with cnfs
either?

open to thoughts...suggestions...discussion?



--
End Of The Line BBS - Plano, TX
telnet endofthelinebbs.com 23

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 05.07.24 09:36, Nigel Reed wrote:

I was thinking about a report I did for my older server, which used the tradspool storage method, and that was to count how many articles are
in a group and list how many new articles have been added since last
run.

Why not use the numbers from active file?

A similar thought that when looking for text, I could grep through a
group or set of groups. Not sure this would even be possible with cnfs either?

That's sadly not possible with cyclic buffers :/
--
.......
Billy G. (go-while)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Thus spake Nigel Reed <[email protected]>

I was thinking about a report I did for my older server, which used the tradspool storage method, and that was to count how many articles are
in a group and list how many new articles have been added since last
run.

Are you familiar with the cnfsstat utility?
It provides part of the information you are looking for:

Class NLS for groups matching "at.*,aus.*,be.*,cz.*ch.*,dk.*,es.*,fr.*,hr.*,it.*,nl.*,no.*,nz.*,pl.*,pt.*,se.*,uk.*,yu.*"
Buffer LNLS00, size: 9.98 GBytes, position: 4.29 GBytes 1.43 cycles
Newest: 2024-07-05 13:03:12, 0 days, 0:00:56 ago
Oldest: 2023-04-02 2:43:08, 460 days, 10:21:00 ago
Buffer LNLS01, size: 40.0 GBytes, position: 1.25 MBytes 1.00 cycles
Newest: 2023-05-01 22:49:34, 430 days, 14:14:34 ago
Oldest: 2023-04-02 22:08:40, 459 days, 14:55:28 ago

I can't see a way to make this happen using cnfs without pulling the
storage token using a similar method as 6.4 in the FAQ and then
manually parsing the newsgroups line from each article.

cnfsstat is written in Perl and is included in the INN sources.
The code may give you an idea of how to retrieve additional information
about the articles stored in CNFS buffers.

--
Пу́тін — хуйло́
https://www.eternal-september.org

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Fri, 5 Jul 2024 11:33:58 +0200
"Billy G. (go-while)" <[email protected]> wrote:

On 05.07.24 09:36, Nigel Reed wrote:

I was thinking about a report I did for my older server, which used
the tradspool storage method, and that was to count how many
articles are in a group and list how many new articles have been
added since last run.

Why not use the numbers from active file?

Because it doesn't take into account deleted articles.

A similar thought that when looking for text, I could grep through
a group or set of groups. Not sure this would even be possible
with cnfs either?

That's sadly not possible with cyclic buffers :/

--
End Of The Line BBS - Plano, TX
telnet endofthelinebbs.com 23

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Hi Nigel,

I was thinking about a report I did for my older server, which used the tradspool storage method, and that was to count how many articles are
in a group and list how many new articles have been added since last
run.

I can't see a way to make this happen using cnfs without pulling the
storage token using a similar method as 6.4 in the FAQ and then
manually parsing the newsgroups line from each article.

You can use overview information to get the number of articles in a
newsgroup.

% getlist -R counts
news.software.nntp 17619 1 13223 y

gives you a list of newsgroups containing the information (13223
articles in news.software.nntp, between article numbers 1 and 17619).
For accurate results, see the groupexactcount setting in readers.conf.
You may want to set it to 0 for connections coming from localhost or
whichever server running your command.

How many articles have been added since the last run is not
straight-forward though... You have to somehow record the latest high
water marks for each newsgroup, and see how many articles have a greater article number than these recorded high water marks.

Use tdx-util or ovsqlite-util for that, assuming you have a tradindexed
or ovsqlite overview.

For instance, assuming I have recorded that the latest highest article
number for news.software.nntp was 17619, I get the number of new
articles with the following command:

% ovsqlite-util -g -n news.software.nntp -a 17620- | wc -l
0


Incidentally, you may also use tdx-util or ovsqlite-util instead of
"getlist -R counts" for the counts:

% ovsqlite-util -i -n news.software.nntp
news.software.nntp 17619 1 13223 y 1720923358 0


I hope this could help.

Do we have any better tools for interacting with cnfs...or if not,
should we?

Naturally, every tool is perfectible and can be enhanced. It just needs someone to write it and share it with the community.

A similar thought that when looking for text, I could grep through a
group or set of groups. Not sure this would even be possible with cnfs either?

You have to force a text search with "grep -a".

--
Julien ÉLIE

« Pour aller plus vite, j'additionne toujours de bas en haut : je fais
du même coup l'addition et la preuve. » (Aurélien Scholl)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On Sun, 14 Jul 2024 09:17:31 +0200
Julien ÉLIE <[email protected]d> wrote:

Hi Nigel,

I was thinking about a report I did for my older server, which used
the tradspool storage method, and that was to count how many
articles are in a group and list how many new articles have been
added since last run.

I can't see a way to make this happen using cnfs without pulling the storage token using a similar method as 6.4 in the FAQ and then
manually parsing the newsgroups line from each article.

You can use overview information to get the number of articles in a newsgroup.

% getlist -R counts
news.software.nntp 17619 1 13223 y

gives you a list of newsgroups containing the information (13223
articles in news.software.nntp, between article numbers 1 and 17619).
For accurate results, see the groupexactcount setting in
readers.conf. You may want to set it to 0 for connections coming from localhost or whichever server running your command.

How many articles have been added since the last run is not
straight-forward though... You have to somehow record the latest
high water marks for each newsgroup, and see how many articles have a
greater article number than these recorded high water marks.

Use tdx-util or ovsqlite-util for that, assuming you have a
tradindexed or ovsqlite overview.

For instance, assuming I have recorded that the latest highest
article number for news.software.nntp was 17619, I get the number of
new articles with the following command:

% ovsqlite-util -g -n news.software.nntp -a 17620- | wc -l
0

Incidentally, you may also use tdx-util or ovsqlite-util instead of
"getlist -R counts" for the counts:

% ovsqlite-util -i -n news.software.nntp
news.software.nntp 17619 1 13223 y 1720923358 0

I hope this could help.

All good information, thank you. this is something that I would run
nightly so I would just need to subtract the previous nights values to
get the number of new articles per group. I'll give it a try tomorrow.
Thanks.

--
End Of The Line BBS - Plano, TX
telnet endofthelinebbs.com 23

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)