The critical need for classifying streaming data arises from its widespread use in real-world industries, where analyzing continuous, dynamic, and evolving data streams accurately and promptly is essential for informed decision-making and gaining predictive insights. Existing research mainly focuses on abundant supervision, overlooking the scarcity and delayed availability of labels, which can vary in timing. Addressing this, the article introduces a new learning method that employs a synchronization-based core support extraction technique. This technique is designed to manage changing concepts and delayed partial labeling by extracting key data points that act as pseudo-labels. Thanks to the concept of synchronization, these extracted key data points accurately represent the inherent local cluster structure in an intuitive manner and better maintain the class structure. Consequently, these pseudo-labels are utilized for classifying future incoming data batches. Furthermore, the method incorporates a knowledge base to summarize and represent all incoming streaming data. Building upon this knowledge base, an ensemble model for classification and an efficient new class detector are proposed. Both operate in a local fashion to ensure robust learning, even in complex class distributions. Evaluations on benchmark datasets reveal a consistent performance lead, surpassing established algorithms by up to 10%, achieving state-of-the-art results. •A new learning method to handle concept drift and delayed labeling.•A novel method for detecting emerging labels.•An ensemble classifier for robust classification.•The proposed method uses a knowledge base to summarize data.•The proposed method is robust and allows yielding state-of-the-art performance.