On 22/06/2025 12:32, Philip Herlihy wrote:
I'm seeing a lot of references to "AI" PCs and laptops, which I
understand have a Neural Processing Module (NPM). I'm finding it hard
to get these in perspective.
Back from my holidays and coming to this thread a little late ...
I'm not an expert ... but I see two ways that people are using AI, at
present.
One is very much cloud-based, and requires little or no assistance from
the client. I'm thinking of things like ChatGPT and its brethren, which
work using huge models that live on the server. NP capability on the
server is presumably useful, but not on the client.
The other is in applications that use pre-built models that can be run
locally. Things like recognition of human forms in images use this, and
they can run much better with GPGPU computing of an NPM.
Aside: I have a cheap (~£20) security camera that has an "AI chip" built
in to help it recognize people, vehicles, pets, etc., as well as just
motion in the images from the sensor. It seems to work quite well, but I
don't know how much the so-called AI chip actually helps, though the CPU
in a £20 camera can't be all that powerful (and presumably the models
for recognizing bipedal and quadripedal forms are easy to bake into a
chip, and simple enough that it doesn't take an expensive chip to run them).
Chip-makers are jumping on the AI bandwagon by adding some NP capability
to CPUs, but only to quite high-end CPUs, as yet. It's not clear to me
whether this is supposed to be helpful with server-side AI applications
like ChatGPT or with client side applications. In either case there
won't be any benefit unless there is software support. Ideally you'd
hope that a standard API for using NP facilities would emerge that would
work with any NP hardware, but the lack of any such API for GPGPU
computing (CUDA is nVidia-only) even now suggests that this may be some
time coming.
So ... I wouldn't worry at this stage about not having AI capability in
a new PC. The playing field is still moving around and we can't see
where the goalposts are going to be, yet. The NP capabilities of a CPU
you buy today may not be relevant in a year or two's time.
In any case, there are NPMs (e.g. Hailo) that can attach to PCIe
(especially M.2) buses. Get one of those, if you're interested enough to
want to play, and you'll be able to replace it with something else when
it becomes obsolete without ditching the entire CPU. If PCs with NPMs
become a thing you can add one of those.
I think the exciting side of AI will be server-side for some time, yet,
though - especially as the AI companies want you to upload your data to
help improve their models (but perhaps I'm a little cynical).
--
Cheers,
Daniel.
--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)