PyoSignal Logo
PyoSignal
Back to Research

Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence

Paper ID: 2607.07675 โ€ข 39 Upvotes
Robot-AI MoE Video-Generation Embodied-AI Video Evaluation Inference Safety
Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence

๐Ÿ“ ํ•ต์‹ฌ ์š”์•ฝ

๋กœ๋ด‡ ์ œ์–ด ๋ฐ ๋ฌผ๋ฆฌ์  ์ƒํ˜ธ์ž‘์šฉ์— ํŠนํ™”๋œ MoE ๊ธฐ๋ฐ˜ ๋Œ€๊ทœ๋ชจ ๋น„๋””์˜ค ์‚ฌ์ „ ํ•™์Šต ๋ชจ๋ธ์ธ LingBot-Video ์ œ์•ˆ

๐Ÿ“– ์ƒ์„ธ ๋‚ด์šฉ

๊ธฐ์กด ๋น„๋””์˜ค ์ƒ์„ฑ ๋ชจ๋ธ์€ ์‹œ๊ฐ์  ํ™”๋ คํ•จ์— ์น˜์ค‘ํ•˜์—ฌ ๋กœ๋ด‡ ์ œ์–ด์— ํ•„์š”ํ•œ ๋ฌผ๋ฆฌ์  ํ˜„์‹ค๊ฐ๊ณผ ํšจ์œจ์„ฑ์ด ๋ถ€์กฑํ•œ ๋ฌธ์ œ๊ฐ€ ์žˆ์—ˆ์Šต๋‹ˆ๋‹ค. ์ด๋ฅผ ํ•ด๊ฒฐํ•˜๊ธฐ ์œ„ํ•ด ๋ณธ ๋…ผ๋ฌธ์€ ๋กœ๋ด‡ ์ง€๋Šฅ์— ์ตœ์ ํ™”๋œ DiT ๊ธฐ๋ฐ˜์˜ LingBot-Video ํ”„๋ ˆ์ž„์›Œํฌ๋ฅผ ์ œ์•ˆํ•ฉ๋‹ˆ๋‹ค. ์•„ํ‚คํ…์ฒ˜ ์ธก๋ฉด์—์„œ๋Š” MoE(Mixture-of-Experts)๋ฅผ ๋„์ž…ํ•˜์—ฌ ๋ชจ๋ธ ์šฉ๋Ÿ‰๊ณผ ์ถ”๋ก  ํšจ์œจ์„ฑ ์‚ฌ์ด์˜ ๊ท ํ˜•์„ ๋งž์ถ”๋ฉฐ ๋Œ€๊ทœ๋ชจ ํ™•์žฅ์„ ์‹คํ˜„ํ–ˆ์Šต๋‹ˆ๋‹ค. ๋ฐ์ดํ„ฐ ์ธก๋ฉด์—์„œ๋Š” ์ธํ„ฐ๋„ท ์˜์ƒ์— ์กฐ์ž‘, ๋‚ด๋น„๊ฒŒ์ด์…˜, 1์ธ์นญ ์‹œ์  ๋“ฑ ๋กœ๋ด‡ ์ค‘์‹ฌ์˜ ๋ฐ์ดํ„ฐ๋ฅผ ๊ฒฐํ•ฉํ•œ ํ”„๋กœํŒŒ์ผ๋ง ์—”์ง„์„ ๊ตฌ์ถ•ํ–ˆ์Šต๋‹ˆ๋‹ค. ๋˜ํ•œ ๋ฌผ๋ฆฌ์  ํ•ฉ๋ฆฌ์„ฑ๊ณผ ์ž‘์—… ์™„์ˆ˜ ๋Šฅ๋ ฅ์„ ๊ฐ•ํ™”ํ•˜๊ธฐ ์œ„ํ•ด ๋‹ค์ฐจ์› ๋ณด์ƒ ์‹œ์Šคํ…œ์„ ๊ฐœ๋ฐœํ–ˆ์Šต๋‹ˆ๋‹ค. ๊ฒฐ๊ณผ์ ์œผ๋กœ ๋ณธ ๋ชจ๋ธ์€ ๋””์ง€ํ„ธ ์ฐฝ์˜์„ฑ์„ ๋„˜์–ด ๋ฌผ๋ฆฌ์  ์‹คํ–‰๋ ฅ์„ ๊ฐ–์ถ˜ ์˜คํ”ˆ์†Œ์Šค ๋น„๋””์˜ค ํŒŒ์šด๋ฐ์ด์…˜ ๋ชจ๋ธ๋กœ์„œ์˜ ๊ฐ€๋Šฅ์„ฑ์„ ์ž…์ฆํ–ˆ์Šต๋‹ˆ๋‹ค.

๐Ÿ”‘ ์ฃผ์š” ๋‚ด์šฉ (Key Points)

  • MoE(Mixture-of-Experts) ์•„ํ‚คํ…์ฒ˜๋ฅผ ํ†ตํ•œ ๋ชจ๋ธ ์šฉ๋Ÿ‰ ํ™•์žฅ ๋ฐ ์ถ”๋ก  ํšจ์œจ์„ฑ ํ™•๋ณด
  • ๋กœ๋ด‡ ๋™์ž‘(์กฐ์ž‘, ๋‚ด๋น„๊ฒŒ์ด์…˜ ๋“ฑ) ์ดํ•ด๋ฅผ ์œ„ํ•œ ํŠนํ™”๋œ ๋ฐ์ดํ„ฐ ํ”„๋กœํŒŒ์ผ๋ง ์—”์ง„ ๊ตฌ์ถ•
  • ๋ฌผ๋ฆฌ์  ํ•ฉ๋ฆฌ์„ฑ ๋ฐ ์ž‘์—… ์™„์ˆ˜ ์ค‘์‹ฌ์˜ ๋‹ค์ฐจ์› ๋ณด์ƒ ์‹œ์Šคํ…œ ๋„์ž…

๐Ÿ’ก ์‹ค๋ฌด์  ๊ฐ€์น˜ (Relevance)

๋กœ๋ด‡ ์ œ์–ด๋ฅผ ์œ„ํ•œ ๋น„๋””์˜ค ์ƒ์„ฑ ๋ชจ๋ธ ๊ฐœ๋ฐœ ์‹œ, ๋‹จ์ˆœ ์‹œ๊ฐ์  ํ’ˆ์งˆ์ด ์•„๋‹Œ ๋ฌผ๋ฆฌ์  ๋ฒ•์น™๊ณผ ๋™์ž‘ ํšจ์œจ์„ฑ์„ ์–ด๋–ป๊ฒŒ ํ•™์Šต์‹œ์ผœ์•ผ ํ•˜๋Š”์ง€์— ๋Œ€ํ•œ ๋ฐฉ๋ฒ•๋ก ์„ ์ œ์‹œํ•ฉ๋‹ˆ๋‹ค.

โœ… ์ถ”์ฒœ ์•ก์…˜ (Actionable Items)

  • ์ œ๊ณต๋œ ๋ฐ์ดํ„ฐ ํ”„๋กœํŒŒ์ผ๋ง ์—”์ง„์˜ ๋ฐ์ดํ„ฐ ๊ตฌ์„ฑ ๋ฐฉ์‹ ๋ถ„์„
  • MoE ๊ตฌ์กฐ๊ฐ€ ๋น„๋””์˜ค ์ƒ์„ฑ์˜ ๋ฌผ๋ฆฌ์  ์ผ๊ด€์„ฑ์— ๋ฏธ์น˜๋Š” ์˜ํ–ฅ ๊ฒ€์ฆ
  • ๋กœ๋ด‡ ์‹œ๋ฎฌ๋ ˆ์ด์…˜ ํ™˜๊ฒฝ์— ํ•ด๋‹น ๋ชจ๋ธ์„ ์ ์šฉํ•˜์—ฌ ๋™์ž‘ ์ƒ์„ฑ ์„ฑ๋Šฅ ํ…Œ์ŠคํŠธ