Topics
The Software Engineering for Machine Learning Applications (SEMLA) initiative aims at bringing together leading software engineers, machine learning experts and practitioners to reflect on and discuss the challenges and implications of building complex data-intensive software systems. The event’s structure follows a working conference where panel, working groups and talks reflect on the many hurdles of developing, deploying and maintaining data-intensive software systems integrating machine learning elements. Our concerns as software engineers can be summarized in the following two important questions:
1) Do data scientists/ML or AI experts/domain experts possess the necessary skills to develop, maintain and extend the complex software implementations of analytical algorithms?
Defining and in vitro validating algorithms is vital, however their implementation and integration in real world applications are equally if not more important, since it’s the software that is going to be used and stressed directly, not the core algorithm. Coding is neither where software begins nor where it ends. Therefore, it is important that the task of implementing the algorithms into software is taken over by well-trained and well-equipped professionals in collaboration with the various experts.
2) Do software engineers have or need to have intimate knowledge of the domain or the algorithm for which they provide an implementation?
Exactly as software skills are important for ML applications, so is the understanding of ML and of the domain. Without it, the software engineer may not include the proper constraints, or develop the proper test cases, or eventually develop the software in an extendable and maintainable manner. While it would be fortunate to have professionals trained in both ML and SE (and possibly be knowledgeable about the application domain), this is rarely possible. Fortunately, software engineering has developed a plethora of techniques and processes to augment communication and collaboration within diverse teams and also between the development team and the domain experts/clients. Unfortunately, these processes are not always adopted by non-software professionals.
In trying to respond to these questions, we may find ourselves circling around two broad solutions, which can lead to different questions on their own:
1) How can we change and shape the education of computer scientists/engineers in order to excel in developing data-intensive software systems (and others of similar nature)?
Nowadays, there is the increasing realization that software engineering and computer engineering need to be multidisciplinary domains given their high degree of application. In this sense, such professionals need to receive broad training and education that, at the same time, is applied and aligns to the needs of the market.
2) What tools do we need to develop to provide support to all stakeholders of such a system (software engineers, ML/domain experts)?
Besides proper training and the formal processes, we also need the proper automated tools to support the development of the system and the communication between the various experts. This will result in efficient and effective development, but it will also increase the success of future extensions. The response to this question will spur new research to facilitate practice.
Specific goals of the symposium include:
- Broadening awareness within the software engineering community of the potential for the application of machine learning algorithms;
- Facilitating the exchange of ideas and interaction between international researchers in machine learning and software engineering;
- Defining open and key research problems faced in realizing usable tools/approaches for intelligent system design, development, test, and evolution;
- Constructing a foundation of materials for future research on SEMLA.
Abstract: Anomaly detection plays an important role in management of modern large-scale distributed systems. Logs, which record system runtime information, are widely used for anomaly detection. However, Unsupervised anomaly detection algorithms face challenges in addressing complex systems, which generate vast amounts of multivariate time series data. Timely anomaly detection is crucial for managing these systems effectively and minimizing downtime. This proactive approach minimizes system downtime and plays a vital role in incident management for large-scale systems. To address these challenges, a method called Multi-Scale Convolutional Recurrent Encoder-Decoder (MSCRED) has been developed for detecting anomalies in CN PTC system logs. MSCRED leverages the power of multivariate time series data to perform anomaly detection and diagnosis. It creates multi-scale signature matrices that capture different levels of system statuses across various time steps. The method utilizes a convolutional encoder to capture inter-sensor correlations and a Convolutional Long-Short Term Memory (ConvLSTM) network with attention mechanisms to capture temporal patterns.
Abstract
Abstract: Language models such as RoBERTa, CodeBERT, and GraphCodeBERT have gotten much attention in the past three years for various Software Engineering tasks. Though these models are proven to have state-of-the-art performance for many SE tasks, such as code summarization, they often require to be fully fine-tuned for the downstream task. Is there a better way for fine-tuning these models that require training fewer parameters? Can we impose new information on the current models without pre-training them again? How do these models perform for different programming languages, especially low-resource ones with less training data available? How can we use the knowledge learned from other programming languages to improve the performance of low-resource languages? This talk will review a series of experiments and our contributions to answering these questions.
