Featured CS Course: Edge Artificial Intelligence (Edge AI)
Training the Next Generation of Experts in AI Efficiency

Since its launch in the second semester of Academic Year 2023, Edge Artificial Intelligence (Edge AI) has been offered for two consecutive years. Although the word "edge" in the course title may sound humorous at first—whether one interprets it as the "wisdom of marginal people," "marginal artificial intelligence," or simply "AI on the edge"—the concept behind it is both precise and profound: AI is rapidly shifting from the cloud to end devices, pursuing higher efficiency, lower power consumption, and real-time on-device computation.

The course is designed by Professor Kai-Chiang Wu, whose long-term research focuses on model compression, system optimization, and AI acceleration. From early convolutional neural networks (CNNs) to the latest Transformer and Mamba architectures, Professor Wu has gradually realized through years of research that having a powerful model is not the hard part—the real challenge lies in making it run well, run fast, and run efficiently. This insight inspired him to transform his research achievements into a course that offers an alternative research direction for CS students who are tired of repetitive model tuning and training but still wish to dive into the world of AI through efficiency-driven innovation.

The Importance of Edge AI

As AI becomes increasingly decentralized, Edge AI is emerging as a major technological trend. Because edge devices operate independently without relying on cloud servers—and without uploading sensitive data—they naturally provide advantages such as enhanced privacy, reduced network reliance, and decreased dependence on large data centers.

However, end devices face significant constraints: limited compute power, insufficient memory, and often only battery-level energy availability. This means software must be lightweight and precise, while hardware must be fast and energy-efficient—principles that lie at the core of the Edge AI course.

Professor Wu's lab simultaneously conducts research on customized AI accelerators and investigates a wide range of model compression and optimization techniques, including pruning, quantization, data tiling, and speculative decoding. With a co-design approach spanning both hardware and software, his team has successfully enabled large models to run efficiently on small accelerators. Multiple research results were accepted to top-tier conferences in 2025, including ICLR, ICML, EMNLP, and NeurIPS, showcasing the team's strong capabilities in AI efficiency and cross-layer integration.

Course Content and Highlights:

Hands-on Depth × Cutting-edge Research = The Ultimate Edge AI Course

The course covers a wide spectrum of technologies, including model compression (pruning and semi-structured sparsity), quantization (post-training and QAT), TinyML, AI accelerators (general-purpose and ASIC), neural architecture search (NAS), knowledge distillation, LoRA/DoRA fine-tuning, exploration of new LLM architectures, and distributed training.

Emphasizing efficiency, the course goes beyond theory by incorporating extensive hands-on practice to help students learn how to make models run better, faster, and with fewer resources—while also sparking further research interest.

In addition, the course features guest lectures from leading experts across industry and academia, including:

• Dr. Cheng-Tao Hsieh (Skymizer) — TinyML
• Prof. Juinn-Dar Huang (NYCU) — AI accelerators
• Dr. Hung-Yueh Chiang (University of Texas at Austin) — Mamba quantization (Quamba)
• Dr. Hao Yu (OpenAI) — FlashAttention and PagedAttention

These lectures expose students directly to state-of-the-art techniques and real-world applications, significantly broadening their perspectives.

Strong Student Response: Enrollment Doubled in Two Years

According to student feedback, the Edge AI course offers rich content and challenging hands-on assignments, making the learning experience highly rewarding. Students report that the course not only deepens their understanding of model mechanics and fine-tuning principles but also gives non-CS students valuable access to the latest developments in Edge AI across academia and industry—including one medical student who noted how inclusive and impactful the course was.

Others highlight that the course's balance of theory and practice allows them to master cutting-edge techniques through actual implementation, providing significant benefits whether they choose to pursue research or enter the tech industry.

In just two years, enrollment in Edge Artificial Intelligence has grown from 70–80 students in its debut year to nearly 160 students in the second year. The course has quickly become one of the most anticipated and well-regarded offerings in the department, demonstrating students' strong interest and enthusiasm for Edge AI and model efficiency technologies.

Beyond equipping students with practical skills in AI optimization, the course opens new pathways for future research and career development.