Looking to build a rig for AI training, could use some advice to that end

Support
1

Just wanna start by saying, please don’t bring up my previous/first post on here. We all have bad days, so let’s just pretend that first post is in a venv and doesn’t affect/infect everything else. Also, apologies if this is in the wrong forum/section, it can be moved if the mods wish to do so.

My primary intention for moving over to Linux, apart from finally embracing open source, and freeing myself from the shackles of corporate ownership (M$), is AI art generation, and AI model training. Although I’ve branched my interests a bit since getting into Linux, these two are still the primary goal for my setup.

I have a middle-end budget for parts(800-1000 or so for mobo, ps, cpu and ram, alredy have case picked out and gpu is on a separate budget), and what I know so far is; AMD is the clear winner for CPU, since AMD fully supports open source, and everything else attaches to the mobo, so I figure AMD is a safe bet for a more stable Linux rig.

In terms of GPU, I’m largely learning towards Nvidia because their GPU’s are geared towards AI application, so in terms of compatibility and speed, Nvidia seems like the best choice, albeit not as friendly with Linux as AMD/Radeon.

Nothing is set in stone yet, I’m trying to get as much unique info as I can before making any final purchases. There’s still outlying variables to consider as well, like, if proprietary Nvidia drivers will be a headache in Linux (I’ve had some on my current system, but I’m running an Aorus laptop, so thats a whole headache in Linux in itself. That said, I won’t buy anything from Gigabyte again). As well as, if open source Nvidia drivers take advantage of CUDA cores, and any other AI based optimisations inherent in Nvidia cards.

If Nvidia, card will be a 4060ti at best, unless I can find a 4070 on a deal. Either way, 16 gig Vram is the realistic goal.

Any input would be appreciated (As long as it’s not akin to ‘google it’ lol…).

Oh, and also, Distros, I do like Manjaro for it’s open-ended customisation, and it’s general navigation, but I am wondering if it’s ideal in terms of stability. So, any pointers there would be helpful as well. Who knows, maybe there’s some AI geared distro out by now. But perhaps too early in AI development for that to be a thing.

2

What AI are you planning to use?

If it’s a cloud one the pc doesn’t really matter, but then you can’t train models.

For stable diffusion 16GiB+ would be preferable (as you already seem to know), especially if you plan on using xl models. I have an RTX4060Ti 16GiB, and it’s fine for that.

From what I remember reading, no consumer grade GPU would really suffice, though you might get away with 24GiB+, but I may be wrong about that. I’ve only read a little on the subject, and it was a while ago.

New GPUs should be coming out around xmas, so you might want to wait and see.

However there are other options, you might want to check out civitai, there’s a couple of guides (for loras at least) there that use Google Colab however you might need a paid account to get the resources required.

Here’s a few links that may (or may not) be useful. I just did a quick search, though I have read some prior, at least partially. You really will need to search for the information.

Dunno, but from what I’ve read most of it is still closed source, just some open source kernel modules to interface with the closed source part.

They have AI cores, but AFAIK you need to use TensorRT.

3

Ist das Kunst oder kann das weg?

(try a translation service from German to your language to decipher that)

How can AI even begin to “create”?

One question is perhaps:
What is art?
Another is what it means to create.

A matter of the definition of the terms …

4

I only have a surface level understanding of what that is, as in I understand the label, and the general thing it refers to, not much else. I seen another post on here about a user that couldn’t get TensorRT drivers working or somesuch. is that something implicitly required for training? I’ve done plenty of generations, but had some headache with cuda crashing when I leave ComfyUI on overnight.

I will look into that though. That said, I prob coulda saved you a lot of time on about 80% of your response if I’d just stated what I usually state when mentioning that I want to get into model training. I want 100% offline, none of that site crap(throwing shade at ‘site crap’ not at you). I want it to be my business what I train and how, no corporate involvement. Ideally I want a 100% offline capable setup. if I was gonna do online, I’d stick with my current 3070ti laptop.

I also, on a personal/idealistic level, don’t really see any merit to those online generators. Anyone’s grandma can use those and get good results.

As for the needing 24gigs to train, that’s a bit of a leetist standpoint (not yours, wherever you read it from). Of course 24+ is the ideal world setting, the more the better, but I don’t live in an ideal world, so I’m limiting card budget to 600ish. Sides, I paid 2k for a laptop last year, and the thing is now considered ‘low end’ for use in AI applications. That’s the last time I blow the bank on video, in a year, it’ll be obsolete.

That said, I’ve heard it’s even possible to train Flux loras on an 8gig card.

So 16 gig should do well for me. especially if I’m just training SDXL.

Also, I have a fair bit of experience already, given that I only started into this a few weeks ago, I’ve already used Easydiffusion, A1111, SD Next, and I’m currently working with ComfyUI. ED was crap compared to Auto, A1111 was subpar compared to SDN, although I do miss the png info tab, that was very useful, and always seemed to work as intended in regards to pulling metadata from the img and sending it where it needs to go, SDN and ComfyUI have similar functions, but, at least in my experience, neither worked as well. Especially comfy with it having it’s own system, workflows, you can’t take an odler image you’ve produced and send it into the txt2img, since it expects a workflow.

That said, as for which AI training applications I’m going to use, that’s very open ended, I installed a lora training widget for comfy, but I haven’t tried it yet besides to pull the widget into the workflow, and disable it until I learn a bit more. Considered kohya, Really digging for a solution that’s 100% offline, and doesn’t send my data all over the place. I don’t arbitrarily fear hackers, it’s more about the corporations. they can stay out my stuff.

so, yea, in terms of training, it’s up in the air atm.

5

Truthfully, a month ago, I woulda agreed with you and moved on. Cuz I had the exact same mindset.

On one hand, AI ‘art’ is a bit shameful, and it takes away from the creativity of humanity. If anything, I should have more bias against it, since I considered myself an artist before AI came along. And was finally beginning to refine my skills, and understanding of the artistic process into something almost marketable. But, along comes AI art, and minimizes the entire human artist struggle, and devalues their hard earned skills.

Now, someone can download a software, insert a prompt, and output art with quality that woulda taken me at least another several years or so to master, if ever.

People are right to be agitated at such a thing. On the other hand, I can rage at the machine, and go down with all those that do, or I can join the enemy, as it were, and get in on the ground floor of a brand new industry.

I could either stand to make some money off simply selling generated ‘art’, before it becomes so automated and optimized that literally anyone with a smartphone can do it, or I can look into that a little, while also learning more about AI in general, (art, and more importantly, AI chat). I plan to look into the latter.

Too many times in my life, I’ve overlooked potential growth in an industry (really ■■■■ the bed when it came to Bitcoin… I could’ve been a millionaire by now if I’d invested what I could have right then). I want to learn every bit about it’s creation, development, applications, etc. And from there, I will find out where I can best fit into all that.

Truthfully, my biggest interest, and where my passion lies, is in digging into the creation, and training of AI models, not AI art, but rather the intelligence models like what ChatGPT uses. I’ve always had a passion for AI, but never the resources that are available for it now. And with getting into Linux, it’s such an easy platform for development, I mean, as an OS, that’s what I truly like about it, it’s an OS that makes software development/testing feel natural, rather than like something that’s alien to the OS, like with how windows is.

1 Like
6

I see @dmt has researched the topic well, and I have little to add…

…except that you should probably increase your expected budget by a factor of ten.

Cheers.

7

Forum Rules - Manjaro

Threads seeking advice about specific computer product recommendations are discouraged.

  • Such topics, like the technology they discuss, quickly become obsolete and are unlikely to provide any lasting benefit to the wider community.
  • You are expected to be able to do your own research and draw your own conclusions about which product best suits your individual requirements.
  • However, Manjaro partners with several hardware vendors. If you’re interested in hardware powered by Manjaro, you are welcome to check out the available options at the bottom of the Manjaro Products page.
2 Likes
8

You forgot the trollface emoji. Although, what you just said would actually go best with the Skeletor meme. I’m well aware that more is better/ideal, but with proper tweaking and optimization, I could do well with a 16 gig desktop card. Hell, I do alright with an 8gb laptop card, I just won’t be able to do top end stuff yet.

Furthermore, I don’t need to spend a boatload on a motherboard, there are 800$+ boards that are geared for AI, but it’s still a motherboad, it’s not gonna make THAT big of a difference in the overall process, the video card is the more important factor. I may go a bit over 1k, I have the bank to do so if needed, but I’m avoiding emptying my entire mutual fund on this.

9

As opposed to the kernel and dependencies upgrading every other week and making most discussions about those obsolete to most people?

I AM doing my own research, by hitting the forums, and seeking individual use data, as opposed to the ‘looks good on paper’ data that can be easily picked up from various webpages, or the sponsored information that makes one product look better than the next just to sell a product. Also, most of the easy access information out there about hardware, and especially in specific use like AI training, does not tailor it’s information to Manjaro, or even to Linux at all.

If I want to know what will work good on windows, I’ll google it, but manjaro is a different beast entirely.

Some actual useful input, I will check this out. Thank you.

10

OK, seriously, is this a joke, or did I miss a link somewhere at the bottom of that page? I don’t want to preemptively come across crass, but if I understand it right, and the “hardware”, you were talking about being listed on the bottom of that page, is just some pre-built laptops and a micro computer.

Then it looks like you’ve just come on here to shut down my discussion about custom computer builds, so you could push Manjaro products on me. Not cool. Not very Linux-minded at all… NGL, this makes me rethink my liking of Manjaro as a primary distro.

If I did just miss a link somewhere and there is indeed a page that lists custom-build hardware(IE: computer parts) that works best with Manjaro, then I’ll take the L for the above paragraph, but if not, I am most ashamed…

11

The last time I went price hunting for hardware suitable for A.I. related activities the average entry level for a GPU alone was around $3,000-$5,000.

Now, this of course would vary according to local currency, market fluctuations in the heat of the pandemic and other factors, but my research at the time showed that a typical system would not generally satisfy the need.

In short, this was a valid observation, though I can understand the need to reject anything that might conflict with your own opinion, or budget.

Now, if I were to choose an emoji; especially after reading your response to my comment, and general attitude toward others; I’d likely look for one with a middle finger inverted.

Letztendlich denke ich, dass A.I. generierte Kunst so alltaglich werden wird, dass sie wegwerfbar wird, wie du vorschlagst.

Mull rein, Mull raus.

Entschuldige das Fehlen von Umlauten… :beers:

12

I quoted one of the rules that apply here because you are a new user and may not be aware that part of OP was discouraged

This discussion has not been closed or unlisted

13

Well, thank you for that, but it seems pretty clear the real reason why it’s ‘discouraged’ (discourage something, then immediately try to sell a product after, lol… totally not sus…).

Just sayin though, I get you’re not in the business of making the rules.

15

Oh, please. You should really read your comments from the point of view of others, sometime. Maybe it would be enlightening; maybe not.

More as the pitty. As you inadvertantly suggest, there has indeed been some profiteering amongst the supposed supply and demand increases resulting from the pandemic period.

Too many after the quick buck ultimately at the expense of others.

Adios.

16

I have, but I wasn’t handled with kid gloves when I was growin up, not even as a child, so irritated words that aren’t even directed at me, do not offend me, why should they? I think it’s this modern generation of people that were getting their hands Purel-ed by mother, as a child every-time they touch a doorknob. So we end up with people that are so sensitive that they end their lives over posts they see on facebook. Not to minimize their suffering, surely they musta been dealing with more in the background, but it does highlight how sensitive people have gotten since internet.

Worst part about that is, corporations use it as an excuse to lock down freedom of speech on most major social media platforms, cuz they can use algorithms now to censor or delete anything that may be offensive to a specific person, or group of people, even if it isn’t objectively inherently offensive in any way. With that, they can censor anything up to and including political opinions.

Ya, I get that this isn’t that, but I’m just talkin big picture, conversely and such. The world would be a better place if people tried harder to understand others instead of merely being offended by anything they don’t understand.

17

@Xephier, you are pretty new here and your posts have been flagged already pretty often for being deemed unprofessional or unwelcome to the community. This is a support forum to help users using the Manjaro distro and adjacent topics, it’s not a chat forum to discuss all kind of projects or ideas you intend to realize. As soon as you become a member there is a member area for more free discussions. So, it’s up to you to rethink your attitude and writing style and time now to read the rules or to be prepared to receive an official warning.

2 Likes