.png.3b3332cc2256ad0edbc2fe9404feeef0.png.8488ab72b8bb2e508209bfe3211b6e08.png)
-
Recently Browsing 0 members
- No registered users viewing this page.
Announcements
-
Topics
-
-
Popular Contributors
-
1
-
2
-
3
-
4
-
5
-
-
Latest posts...
-
76
Promises made Promises kept , POTUS first few days! The lefts radical response!
Because, as they represent around 40% of agricultural workers, and employers may find it difficult to replace them if they are expelled? -
9
Chinese start-ups such as DeepSeek are challenging global AI giants
The above video by Microsoft engineer David Plummer is a clear intro to what Deepseek is, and isn't. The clever part is its distillation process to turn larger models into smaller, good enough, models that use less hardware. But I do see a possible cat in the bag and asked Grok to confirm (we're already a mix of human and AI!). I post my question and it's answer for clarity. However it goes, Deepseek's arrival will be transformative. Me: My question is about the newly released Deepseek AI software and its operation. The question is specific. Preface: Deepseek is said to use larger AI models to train a number of smaller models that can run on smaller hardware and thus cheaper. Does this mean that 1) larger hardware and training is still required when training the smaller modules? and 2) does the Deepseek open source code they released include the initial large training model, or it only includes code for running the smaller models? (Spoiler, required and no not released) Grok: Larger Hardware and Training for Smaller Modules: Yes, the process of training smaller models through distillation or transfer learning still requires the use of larger hardware and computational resources initially. DeepSeek uses a large model (like DeepSeek-R1 or DeepSeek-V3) to train these smaller models. The initial training of the large model necessitates significant computational resources, but once trained, the knowledge can be distilled into smaller models that require less computational power for deployment. This is evident from the strategies mentioned where DeepSeek-R1 knowledge is distilled into smaller models, which implies that the heavy lifting is done during the training of the larger model. Open Source Code and Model Inclusion: The open-source release by DeepSeek includes models that are distilled from the larger models, particularly the DeepSeek-R1 series. However, the training code for the initial large models (like DeepSeek-V3 or the base models used for distillation) does not seem to be open-sourced. The comments on X and technical reports indicate that while the distilled models and possibly some of their training code are available, the actual training pipeline for the larger models, including the data and the original training code, are not part of the open-source release. This means users can run and possibly fine-tune the smaller models but cannot replicate the training of the large models without additional resources or information. In summary, while you can utilize and potentially further train the smaller models with less hardware, the groundwork of training the larger models still requires substantial resources. The open-source materials provided by DeepSeek focus on the smaller, distilled models rather than the full spectrum of their training process for the large models. End of Grok. -
54
A rant about Thai driving.. I know.. Yaaaawwwnn
The first company vehicle I got, I was required to pass a defensive driving course. That has been a useful skill in Thailand. I just assume everyone else on the road is an idiot trying to kill me, and drive accordingly. Advice to the OP - take up golf. No cars or scooters there, except for the ground staff, who will bow to you, and wai. It's a walk of 7-8 km, with no uneven pavements to trip over. -
142
-
962
-
17
No alcohol Friday
I think you missed "the day I go out drinking" for many it is a social event and not a chance to stock up and sit at home with a few cans.
-
-
Popular in The Pub
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now