1. Forum
  2. Usenet
  3. COMP.AI

  • Is it possible to train generative AI on CPU only?

    From Carl Fink@21:1/5 to All on Thu Nov 28 14:05:34 2024
    I have a system with a very high-end CPU (for a workstation), a 12th
    generation Intel i9, but a low-end GPU (Radeon RX470 with 6 gb of RAM).

    Is it possible to do any sort of AI training on that setup, say if you're willing to process at 5% of an nVidia GPU's speed? I just want to play with
    it, see if it's worth investing more money, at this point. I don't mind, if, say, training a LORA takes an hour per photo, or even 6 hours per photo.

    Sorry if I mangled any terminology, this is not my field and I'm (as you guessed by now) just starting to learn it.

    Thanks.
    --
    Carl Fink carl@finknetwork.com https://reasonablyliterate.com https://nitpicking.com If you want to make a point, somebody will take the point and stab you with it.
    -Kenne Estes

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Tristan Miller@21:1/5 to Carl Fink on Sat Nov 30 23:56:05 2024
    Greetings.

    On 2024-11-28 13:05, Carl Fink wrote:
    I have a system with a very high-end CPU (for a workstation), a 12th generation Intel i9, but a low-end GPU (Radeon RX470 with 6 gb of RAM).

    Is it possible to do any sort of AI training on that setup, say if you're willing to process at 5% of an nVidia GPU's speed? I just want to play with it, see if it's worth investing more money, at this point. I don't mind, if, say, training a LORA takes an hour per photo, or even 6 hours per photo.

    Yes, it's absolutely possible to train generative AI on a CPU rather
    than a GPU, but whether this is feasible depends entirely on what sort
    of model you are expecting to end up with, how much training time you're willing to tolerate, and how much RAM you have available. Without
    further details on your requirements it's difficult to make any specific recommendations. I've seen a few reports comparing CPU and GPU training
    of LoRA models for Stable Diffusion, for example, that indicate that CPU training can require four times more memory and/or forty times more
    time. If you don't already have the requisite GPU hardware and don't
    want to buy it yourself, it might be economical to rent the processing
    power from a cloud service.

    Regards,
    Tristan

    --
    =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
    Tristan Miller
    Free Software developer, ferret herder, logologist
    https://logological.org/ =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Carl Fink@21:1/5 to Tristan Miller on Sun Dec 1 20:23:00 2024
    On 2024-12-01, Tristan Miller <psychonaut@nothingisreal.com> wrote:

    Yes, it's absolutely possible to train generative AI on a CPU rather
    than a GPU, but whether this is feasible depends entirely on what sort
    of model you are expecting to end up with, how much training time you're willing to tolerate, and how much RAM you have available. Without
    further details on your requirements it's difficult to make any specific recommendations. I've seen a few reports comparing CPU and GPU training
    of LoRA models for Stable Diffusion, for example, that indicate that CPU training can require four times more memory and/or forty times more
    time. If you don't already have the requisite GPU hardware and don't
    want to buy it yourself, it might be economical to rent the processing
    power from a cloud service.

    Can you link to those reports, or tell me where to search (e. g. a journal name)?

    This box has 48 GB of RAM. As I wrote originally, I'm fine with long
    processing times, although 40x seems very long.

    Thank you.
    --
    Carl Fink carl@finknetwork.com https://reasonablyliterate.com https://nitpicking.com If you want to make a point, somebody will take the point and stab you with it.
    -Kenne Estes

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Tristan Miller@21:1/5 to Carl Fink on Mon Dec 2 18:46:53 2024
    Dear Carl,

    On 2024-12-01 19:23, Carl Fink wrote:
    Can you link to those reports, or tell me where to search (e. g. a journal name)?

    This box has 48 GB of RAM. As I wrote originally, I'm fine with long processing times, although 40x seems very long.

    I'm afraid I was reporting only half-remembered results from the last
    time I looked into the question, which would have been months ago. I
    did a quick web search just now and came up with a couple queries from
    the Kohya's GUI GitHub project that roughly accord with my recollection:

    https://github.com/bmaltais/kohya_ss/discussions/679 https://github.com/bmaltais/kohya_ss/issues/2632

    The first of these claims CPU training time of about 40×, and the second claims 4× for both time and memory. They both refer to Stable Diffusion.

    There's also this LoRA finetuning guide for LLaMA that provides detailed
    CPU time and memory metrics for various models: https://rentry.org/cpu-lora

    Regards,
    Tristan

    --
    =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
    Tristan Miller
    Free Software developer, ferret herder, logologist
    https://logological.org/ =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)
  • From Carl Fink@21:1/5 to Tristan Miller on Wed Dec 4 09:47:36 2024
    On 2024-12-02, Tristan Miller <psychonaut@nothingisreal.com> wrote:

    I'm afraid I was reporting only half-remembered results from the last
    time I looked into the question, which would have been months ago. I
    did a quick web search just now and came up with a couple queries from
    the Kohya's GUI GitHub project that roughly accord with my recollection:

    https://github.com/bmaltais/kohya_ss/discussions/679 https://github.com/bmaltais/kohya_ss/issues/2632

    The first of these claims CPU training time of about 40×, and the second claims 4× for both time and memory. They both refer to Stable Diffusion.

    There's also this LoRA finetuning guide for LLaMA that provides detailed
    CPU time and memory metrics for various models: https://rentry.org/cpu-lora

    Thank you!
    --
    Carl Fink carl@finknetwork.com https://reasonablyliterate.com https://nitpicking.com If you want to make a point, somebody will take the point and stab you with it.
    -Kenne Estes

    --- SoupGate-Win32 v1.05
    * Origin: fsxNet Usenet Gateway (21:1/5)