Akash Gaurav is a Senior Principal Software Engineer at Cadence Design Systems in Burlington, Massachusetts (USA), a leading Electronic Design Automation (EDA) and Intelligent System Design provider delivering hardware, software, and Intellectual Property (IP) for electronic design.
Akash applies his advanced engineering expertise to developing and optimizing high-performance emulation hardware and software solutions for supercomputing systems. At Cadence, he specializes in architecting and implementing complex algorithms, compilers, and electronic databases to support scalable, large-scale chip design verification for Artificial Intelligence (AI), Machine Learning (ML), mobile, Internet of Things (IoT), automotive chip modules, Graphics Processing Units (GPU), and custom processors.
Akash has spent the last decade gaining extensive experience in C++ chip emulation platforms and HDL synthesis, and has achieved significant success in designing high-performance distributed systems and emulation platforms. TechBullion spoke with Akash about how he is working to overcome scalability issues in AI chip design with solutions that enable the semiconductor industry to bring AI chips to market faster, driving innovation.
Q: Akash, tell us about your background and your current role. Why have you focused your software engineering expertise in chip design, and specifically in high-performance distributed systems and emulation platforms?
A: I’m currently a Senior Principal Software Engineer at Cadence Design Systems in Burlington, MA, where I’ve been working since 2017. My academic background is in computer science. I have a master’s degree from Texas A&M University and a bachelor’s from Nagpur University in India.
Throughout my career, I’ve focused on chip design, particularly in high-performance distributed systems and emulation platforms, because of the exciting challenges they present. The complexity of modern chip design requires innovative solutions in software engineering to handle massive amounts of data and perform intricate computations efficiently.
At Cadence, I’ve been deeply involved in developing critical components of the Palladium Z emulation hardware, including the operating system, compiler, and interconnects. This work allows me to leverage my expertise in C++, distributed systems, and algorithm design to push the boundaries of what’s possible in chip emulation and verification.
I’m particularly drawn to this field because it sits at the intersection of software and hardware, requiring a unique blend of skills to optimize performance at both levels. The constant evolution of semiconductor technology also means there are always new challenges to tackle, which keeps the work intellectually stimulating.
Q: How does your work relate to AI tools and the larger semiconductor industry?
A: My work is closely tied to AI tools and the broader semiconductor industry in five critical ways. First, in emulation platforms: the Palladium Z emulation hardware I work on is crucial for verifying and validating complex chip designs, including those used in AI and machine learning applications. These platforms allow designers to test and optimize their chips before manufacturing, which is essential for the development of AI-capable semiconductors.
Second, algorithm design: I have designed various performance-critical, graph-based algorithms and combinatorial algorithms, which are fundamental to many AI and machine learning tasks. This work contributes to the efficiency of chip designs used in AI applications.
Third, distributed systems: the “in-memory” distributed system I developed to handle terabytes of data is essential for managing the massive datasets required for training and running large AI models.
Fourth, performance optimization: my expertise in multi-threading, concurrency, and asynchronous programming directly contributes to the performance optimization of chips used in AI and other computationally intensive applications.
Fifth, compiler development: the compiler work I’ve done is crucial for translating high-level AI algorithms into efficient machine code that can run on specialized AI hardware. In the larger semiconductor industry, my work contributes to the development of more powerful, efficient, and reliable chips. This is particularly important as the demand for AI-capable hardware continues to grow, driving innovation in the semiconductor sector.
Q: You are currently working on overcoming scalability challenges in HDL synthesis techniques for AI chip design. What are the obstacles to scalability, and how do they impact chip production and the end user market?
A: In my recent work on HDL synthesis for AI chip design, I’ve identified several key obstacles to scalability:
1) Size and complexity: AI chip designs are extremely large and complex compared to traditional chip designs. This makes it challenging to synthesize them using conventional centralized methods.
2) Memory constraints: traditional HDL synthesis tools often struggle with the memory requirements of large AI chip designs. The entire design needs to be “in memory” (RAM) to work, which becomes unfeasible as designs grow larger.
3) Data dependencies: unlike software development, the synthesis step cannot be entirely distributed due to data dependencies and consistency issues. Analyzing and transforming a highly interconnected description of hardware often requires global design knowledge.
4) Sequential processing: many steps in the synthesis process are sequential and hard to parallelize. For instance, incremental elaboration is only possible through progression down the hierarchy.
These obstacles impact chip production, innovation, and the end-user market, resulting in time-to-market delays and increased costs, among other problems. The challenges in synthesizing large AI chip designs can slow down the overall chip development process, delaying the release of new AI chips to the market, and the need for more powerful computing resources to handle synthesis can drive up development costs, which may be passed on to end-users. There are also performance trade-offs, because difficulties in optimizing large designs may result in chips that are less efficient, in terms of power consumption or performance, affecting the end-user experience. I’ve also seen that scalability challenges can constrain designers’ abilities to implement cutting-edge AI architectures, potentially limiting the capabilities of AI chips available to end-users.
These scalability challenges must be addressed to enable the development of more advanced AI chips, and to make advanced chips more accessible to a broader market. Increasingly innovative technologies depend on the design and availability of chips that are equal to the required tasks.
Q: You recently developed a new solution for the problem of scalability in HDL synthesis. Before we discuss your innovation, can you briefly describe HDL synthesis for us?
A: HDL (Hardware Description Language) synthesis is a critical process in chip design that transforms a high-level description of a digital circuit into a gate-level representation that can be implemented in hardware. The key steps in HDL synthesis are:
1) HDL import/reading: the synthesis tool loads the HDL source file that describes the design in terms of logic gates, wires, registers, memory, and other hardware components.
2) Elaboration: this step expands the HDL into a more detailed intermediate representation, resolving references and dependencies.
3) Optimization: various techniques are applied to improve design performance, power consumption, area, thermal management, reliability, fault tolerance, timing closure, and signal integrity.
4) Technology mapping: the low-level design is mapped to targeted hardware, either ASIC (Application-Specific Integrated Circuit) or FPGA (Field-Programmable Gate Array).
Traditionally, HDL synthesis is performed on a single machine with high computing resources such as RAM and CPU. This centralized approach works well for regular chip designs but faces significant challenges when dealing with the scale and complexity of modern AI chip designs. The synthesis process is critical because it bridges the gap between the abstract, human-readable hardware description and the concrete, implementable circuit design. It allows designers to work at a higher level of abstraction while ensuring that their designs can be realized in actual hardware.
In the context of AI chip design, HDL synthesis becomes particularly challenging due to the scale and complexity of modern AI architectures. This is where innovations in synthesis techniques become crucial for improving scalability and efficiency.
Q: Your solution for HDL synthesis is called Distributed Memory Architecture. Explain how this solves the problem, and how it compares to other scalability solutions. What led you to design this solution? What advantages does it offer for AI chip design?
A: The Distributed Memory Architecture (DMA) solution I developed addresses the scalability challenges in HDL synthesis for AI chip designs. What makes this solution unique is the Custom Resource Manager: I created a specialized “Resource and Memory” manager that allows for custom memory allocation-deallocation and custom classes to hold different objects and collections of HDL. This utilizes three features, starting with memory pools. The manager assigns distinct memory pools to different object categories, enhancing performance by reducing system call overhead. Then, when local memory is insufficient, the system offloads memory to remote systems and loads it on demand over the network—this is the distributed memory that differentiates this solution. Smart offloading is built into the system, so that memory offloading is based on least used data, usage-driven offloading, or historical data using machine learning predictions.
The main alternative to DMA is a partition-based approach, which divides designs into modules that can be synthesized independently. However, this method has limitations, because it t may not work for designs that aren’t easily partitionable, it struggles with sequential logic with tight timing constraints, and it is less effective for designs relying on emergent behavior or analog and mixed-signal designs. In contrast, the DMA is more versatile and scalable, adapting to various HDL design types.
What led me to develop this solution is the increasing complexity of AI chip designs, which push the limits of traditional centralized synthesis methods. We needed a solution that could handle designs far larger than available local memory, while maintaining performance and consistency. The advantages of DMA include scalability, flexibility, performance, and futureproofing. It can efficiently handle increasing design complexities which meets AI chip design need, and it adapts to various HDL design types, unlike partition-based approaches. Plus, by optimizing memory usage and distribution, it maintains high performance even with extremely large designs. DMA also aligns well with the trend towards more complex and larger AI models, offering a solution that can grow with advancements in AI technology.
This solution significantly improves the overall scalability and efficiency of the synthesis process, enabling semiconductor companies to bring more complex and powerful AI chips to market faster. This ultimately drives innovation in the AI industry.
Q: How do you think the enactment of the CHIPS and Science Act on August 9, 2022, has impacted the U.S. semiconductor industry? In the two years since the Act was passed, have you seen demand increase for complex AI chips?
A: While my work focuses more on the technical aspects of chip design rather than policy, based on my industry experience and the trends I’ve observed, I believe the CHIPS and Science Act has and will continue to drive increased investment in the semiconductor industry. The Act provides substantial funding and tax incentives for semiconductor research, development, and manufacturing in the U.S., which further stimulates investment and creates jobs. I also see increased emphasis on developing cutting-edge technologies, particularly in AI chip design, which aligns with my work on scalable HDL synthesis for AI chips. Additionally, I am excited to see the boost the CHIPS Act has given to research and development. The focus on expanding R&D in the semiconductor industry and especially in AI has likely accelerated innovation in these fields.
All of this is contributing to a noticeable uptick in the demand for complex AI chips. This is evident from the increasing complexity of designs we’re working with in the emulation platforms. I have also seen a trend towards more sophisticated and specialized AI chip designs, requiring more advanced design and verification tools. This increase in design complexity aligns with the scalability challenges I’ve been addressing in my work.
From my perspective, production is not keeping pace with chip demand, primarily due to a fewer number of highly specialized chip fabrication facilities (“fabs”). Building and equipping a new fab can take years, but I do think the CHIPS Act is facilitating fab construction and expansion to help close that gap in the U.S.
Q: Where do you foresee the future of AI and chip design going? What new obstacles do you anticipate as new applications for AI are introduced, and are you already envisioning solutions to these new challenges?
A: Based on my experience in chip design, I foresee several trends and challenges in the future of AI and chip design. First, we will see increasingly complex AI architectures. As AI models grow larger and more sophisticated, chip designs will need to accommodate these complexities. We will also likely see more application-specific AI chips optimized for tasks or domains. Then, to overcome physical limitations, multi-chip modules and 3D stacking will become more prevalent. Next, there is a growing emphasis on AI chips for edge devices, requiring designs that balance performance with power efficiency. Finally, we may see hybrid systems that combine classical AI with quantum processing for specific applications.
In terms of obstacles, I anticipate ongoing challenges in scalability in design and verification. As designs grow more complex, current tools and methodologies may struggle to keep up. Power efficiency may also be a problem, as it will be very challenging to manage power consumption in increasingly complex AI chips. We will also have to overcome issues in thermal management, to deal with heat dissipation in more densely packed, high-performance AI chips. Complex AI chips will additionally require more sophisticated verification methods to ensure correctness and reliability. Last but not least will be manufacturing precision: maintaining yield and reliability at smaller process nodes will be challenging.
Because all of these problems are inherent to my work, I spend a lot of time thinking about solutions to current and future problems. I am excited about the possibilities for AI-assisted design and verification, in which we leverage AI capabilities to help design and verify AI chips, potentially creating a positive feedback loop of innovation. I think there is also promise in exploring novel cooling technologies to address thermal management issues. I am also interested in hierarchical verification approaches, through which we could develop methods to verify complex systems at multiple levels of abstraction.
Q: As a Senior Principal Software Engineer, how do you lead and mentor your IT teams? What kinds of specialized technical and other skills do you think young engineers will need to keep up with and ahead of current and future industry challenges?
A: My approach to leading and mentoring IT teams is based on fostering a culture of continuous learning, collaboration, research, learning new trends, and innovation. Staffing continues to be a challenge in the semiconductor industry, so training and advancing young engineers in their careers is critical to meeting our present needs and positioning the U.S. industry for growth. When I mentor students and software engineers, I encourage them to pursue the following 10 areas:
1) Strong foundation in computer science: a deep understanding of algorithms, data structures, and system design principles is crucial, especially for developing efficient HDL synthesis tools.
2) Proficiency in relevant programming languages: C++, C, Python, and hardware description languages like Verilog are particularly important in our field.
3) An understanding of parallel and distributed computing is essential for handling large-scale problems in chip design and verification, as demonstrated by my Distributed Memory Architecture solution.
4) Knowledge of Machine Learning and AI: Understanding both the principles of AI and how they translate to hardware design is increasingly important.
5) Hardware design principles: familiarity with digital logic design, computer architecture, and ASIC/FPGA design flows.
6) Performance optimization skills: the ability to profile and optimize code for high-performance applications is crucial for efficient HDL synthesis.
7) Cloud computing and distributed systems: obtain skills in technologies that enable scalable, distributed solutions.
8) Version control and CI/CD: develop proficiency with tools like Git, along with an understanding of continuous integration/continuous deployment practices.
9) Soft skills including communication, collaboration, adaptability, and problem solving. Engineers must have the ability to clearly explain complex technical concepts, such as HDL synthesis techniques, and be able to work in cross-functional teams, bridging hardware and software domains. The field is rapidly evolving, so being able to learn and adapt quickly is crucial. To succeed in this field, you must also be able to approach complex issues, like scalability in AI chip design, with creativity and persistence.
10) Domain-specific knowledge: advance your understanding of semiconductor design processes and challenges specific to AI chip design and verification.
By focusing on these areas, young engineers can position themselves to tackle current challenges and adapt to future developments in the semiconductor and AI industries. As a mentor, I strive to guide team members in developing these skills through hands-on experience with cutting-edge projects, continuous learning opportunities, and exposure to the complex challenges we face in AI chip design and verification.
