I imagine there are quite a few people on here with AI projects.  I am not very experienced in it but I have started to learn.  What are you working on?

Personally, I am going the private route as I imagine the data mining will be prevalent, censorship and bias are high in "Open"AI and I also think down the road there will likely be intellectual property disputes or at least the potential if you develop anything using their tools.  

LLM related tools I have tried:

• LM Studio - Impressive but there are some things in their TOS that makes me think privacy may not be as important to them as they claim.


• PrivateGPT - works pretty well and I get the impression that privacy is actually important to that project.  It is not nearly as refined but there is very active development and it works.


• I also followed along with a multi-modal LLM project by Leon Sander but it looks like he doesn't have long-term plans for it.  I think his general idea was great though.

I have some machine learning projects with work that I can't get into too much detail here but basic regression stuff.  

What projects do you have going on?  What projects are you following?

What hardware are you running it on?

@right_rudder
Just a PC:

7800x3d
128GB RAM
4070 TI Super

Right now it runs well enough for most things but ultimately if I want it or anything for that matter to be augmented with massive data for work or my personal uses I need to work on pre-ingesting the data and being able to store it on the drive, query it and load what needs to be in loaded into VRAM.  Otherwise the that becomes a limitation.

That's strong enough to run just about anything out there. LM studio is what I use that's off the shelf as they say.

what was in the TOS that you objected to?
I've never looked but I can't imagine what it would be sending off device.

There is a paragraph in there that states they can inspect, monitor and I think report how their users use their product.  Something along those lines.  It wasnt too clear and the ambiguity is what put me off.

I'm not doing anything wrong but that isn't the point.  If there is more clarity added I may not have an issue with what they actually mean but if it just becomes a data mine well then it is a deal breaker.

I still use it to try something stuff out as it is a good product but the project I want to complete would have access to a ton of my data (tax docs, intellectual property, etc)  in vectors I wouldn't want to share with anyone.

It may be nothing to be concerned with but I want to make sure before I open that door.

Right now I have a working setup but everything goes straight to vram which is great for many session scenarios but I want what I think is considered RAG or maybe a variation of it.

I am moving through that slowly as most of the tutorials and projects for that are leveraging "open"AI.

I am new to all of this and don't have a programming background.  Just a guy with ideas.  I see this as the largest disruption to the status quo and I'd love to turn some of these leftist zombie corps into circuit city, blockbuster or sadly sears.

I gotcha. I'd start with playing with LM studio if you haven't already.
there's another option but I can't think of the name at the moment but it sounds like what you're looking for is a tool to make your own model.

Everything I've used to make models and training is Mac only so I'm not sure what's out there ( but positive there is options, just not sure how easy to use).

I can say if you give it another 12 to 18 months there's going to be a new revolution (man I hate that over used term) in AI.
Right now everyone is wrapped up in the hype but it's using already existing models that are general and broad.
the next phase is going to be tools to make it easier for the user to make customer specific models.
That's coming soon but the work has just started so it's going to be a bit.

llamafile is the quick and easy answer to get started.  

https://github.com/Mozilla-Ocho/llamafile

I've used privategpt and lm studio as well but you can't beat llamafile for simple and just works everywhere.