Shelf monitoring plays a key role in optimizing retail shelf layout, enhancing the customer shopping experience and maximizing profit margins. The process of automating shelf audit involves the ...detection, localization and recognition of objects on store shelves, including diverse products with varying attributes in unconstrained environments. This facilitates the assessment of planogram compliance. Accurate product localization within shelves requires the identification of specific shelf rows. To address the current technological challenges, we introduce “Shelf Management”, a deep learning-based system that is carefully tailored to redesign shelf monitoring practices. Our system can navigate the complexities of shelf monitoring by using advanced deep learning techniques and object detection and recognition models. In addition, a complex semantic module enhances the accuracy of detecting and assigning products to their designated shelf rows and locations. In particular, we recognize the lack of finely annotated datasets at the SKU level. As a contribution to the field, we provide annotations for two novel datasets: SHARD (SHelf mAnagement Row Dataset) and SHAPE (SHelf mAnagement Product dataset). These datasets not only provide valuable resources, but also serve as benchmarks for further research in the field of retail. A complete pipeline is designed using a RetinaNet architecture for object detection with 0.752 mAP, followed by a Deep Hough transform to detect shelf rows as semantic lines with an F1 score of 97%, and a product recognition step using a MobileNetV3 architecture trained with triplet loss and used as a feature extractor together with FAISS for fast image retrieval with an accuracy of 93% on top-1 recognition. Localization is achieved using a deterministic approach based on product detection and shelf row detection. Source code and datasets are available at https://github.com/rokopi-byte/shelf_management.
•Automated shelf audit process.•Key steps: shelf row detection, product detection, identification.•Efficient and accurate evaluation through advanced deep learning.•Streamlined process, improved efficiency, valuable insights for retailers.
A product recognition system recognizes all products on the shelf images and determines their positions. A business equipped with an automatic product recognition system has a convenient follow-up of ...many human-powered activities while increasing customer satisfaction. That is, product recognition stands out with its benefits such as tracking shelf layouts and stocking their status, and improving the shopping experience for customers, especially the visually impaired ones. However, product recognition is a challenging problem of computer vision in terms of the difficulty of obtaining and updating datasets and the breadth of the product scale. On the other hand, the number of studies on product recognition is constantly increasing by using various computer vision and machine learning methods, and effective solutions are offered to this problem. This paper provide a comprehensive review in the field of researches of product recognition on grocery store shelves. In this article, data sets and approaches used in the literature for the development of an automatic product recognition system are examined and compared, and their benefits and limitations are commented. Finally, a guideline is provided for future researchers and new perspectives for future studies are presented.
Display omitted
Currently, the retail sector is quite competitive. The proper placement of products and giving them shelf space is crucial to the industry's success. On planogram, in order to achieve better ...visibility, the products may be placed on multiple shelves on multiple orientations. The main purpose of this work is to develop and solve a shelf space allocation problem that takes into account shelf, product, orientation, multi-shelves allocation factors. These reflect the realistic conditions of the retail environment. In this paper, a new approach has been used to transform a non-linear shelf space allocation problem into a linear one. We explore how to adopt a more strategic approach of replacing the product of two binary variables and a product of binary and continuous variables in the constraints imposed by the retailer. This allows achieving the optimal solution of product on shelves allocation with the goal of profit maximization. The experiments were performed with the help of a commercial CPLEX solver.
•A mismatch between product shape and rack dimensions can cause suboptimal planograms.•Existing approaches consider that the shelves on a planogram as fixed length, which is limiting.•We propose a ...novel model that jointly determines the shelf dimensions and product allocations.•Two metrics are introduced to quantify product variation and space fitness.•A novel decomposition-based approach using particle swam optimization and constraint programming is proposed.•Key insights are derived from testing of our approach on realistic data; 35,139 planograms from 109 retail stores and 32,399 unique products.•A case study that optimizes a real planogram at a nearby store is also presented to illustrate the usefulness of our approach.
Effective design of planograms in a retail store can improve the visual representation of products on shelves. Existing approaches, however, assume that the shelf length and height are fixed, which can result in unused space on the planogram and suboptimal assignment of product facings, both resulting in reduced revenue for the retailer. To address this real-world challenge, we introduce the joint shelf design and shelf space allocation (JSD-SSA) problem, which determines the optimal shelf design along with product placement considering product family constraints. We propose a decomposition-based approach, which first partitions the planogram area considering product families and then allocates products in that family to the partitioned area. Our novel hybrid approach relies on Particle Swarm Optimization for partitioning and Constraint Programming for product assignment. We also propose two metrics; one to measure variation in product shapes within and between product families, and another to measure the space tightness of a planogram. Experiments indicate that shorter shelf lengths can increase retailers’ profit by up to 37% depending on the product-family shape variation. Higher within-family shape variation can result in higher revenue. Additionally, if product and planogram dimensions share a common factor or multiple, then more compact planograms can be designed, in turn reducing unused space and increasing a retailer's profit. Our approach is general enough to handle gondola, peg, bin, and mixed type of shelving typically found in retail stores.
The ability to recognize a product on the shelf of a retail store is an ordinary human skill. The same recognition problem presents an exceptional challenge for machine vision systems. Automatic ...detection of products on the shelf of a retail store provides enhanced value-added consumer experience and commercial benefits to retailers. Compared to machine vision based object recognition system, automatic detection of retail products in a store setting has lesser number of successful attempts. In this paper, we present a survey of machine vision based retail product recognition system and define a new taxonomy for this field. We also describe the intrinsic challenges associated with the problem. In this comprehensive survey of published papers, we analyze features used in state-of-the-art attempts. The performances of these approaches are compared. The details of publicly available datasets are presented. The paper concludes pointing to possible directions of research in related fields.
Display omitted
•Product detection boosts value-added consumer experience & commercial benefits to retailers.•We present a survey on computer vision methods for detection of retail products.•We analyze the features used in detecting retail products.•We propose a taxonomy of state-of-the-art methods in detecting retail products.•We present challenges and new-frontiers in detecting products.
•Impacts of non-periodic are similar to harmonic current but more difficult to mitigate.•Time-frequency distortion index is most suitable for non-periodic loads.•Hybrid time-frequency method provides ...temporally distributed current reference.•Temporal distribution of compensation increases efficiency and reduces overall cost.•Fuzzy algorithm based on non-periodic characteristics decreases bandwidth requirements.
Retail shelves are adjustable by varying the number of shelf boards as well as the height and depth of each shelf board. Shelf planners adjust the boards accordingly at regular intervals when they create the shelf plans and allocate products. Current shelf planning models assume given shelf configurations and allocate only products. However, the dimensioning of a shelf segment and product allocation are interdependent. For instance, the height of one segment may be reduced if only small products are allocated or products cannot be stacked. This paper proposes the first integrated approach for shelf segment dimensioning and product allocation. It jointly determines the number of facings for each product, the shelf quantity and the size and number of shelf segments. We also identify and consider several restrictions for the shelf structure (e.g., technical options), allocation rules (e.g., maximum inventory reach) and allocation- and shelf-layout-dependent demand. We formulate the decision problem at hand which is an Integer Non-linear Program and apply a solution algorithm based on the application of bounds that are obtained by transferring constraints to a preprocessing stage. Doing so, we can reformulate the problem as Binary Integer Program, provide an exact approach and generate practical applicable and optimal solutions in a time-efficient manner. We show that integrating shelf dimensioning into product allocation results in up to 5% higher profits than benchmarks available in literature. By means of a case study we show how planning can be improved, and that the retailer’s profit margin can be improved by up to 7%.
In this research, we proposed a practical shelf space allocation model with the vertical and horizontal categorization of products. Five groups of constraints, such as shelf, product, multi-shelves, ...orientation and band constraints, are implemented in the model. We proposed two heuristics to solve the retailer's profit maximization problem. The experiments were performed on small instances, and the solution was compared to the optimal solution found by the CPLEX solver. The experiments proved that heuristics could find the optimal solution in most cases (14/20 for heuristics H1 and 16/20 for heuristics H2) without checking the whole solutions space. The lowest solution quality is 97.68% which shows that the heuristics allow finding high-quality near-optimal solutions. Therefore the steps implemented in the proposed heuristics should be used in more complicated shelf space allocation problems for which it is impossible to find a solution by the solvers.
In the ever-evolving landscape of retail, understanding shopper behavior is pivotal for optimizing sales and effectively managing product availability and placement. This study explores the ...integration of autonomous mobile robots into the shelf inspection process, leveraging advancements in automation, information, and robotics technology. Performing mapping tasks, these robots incorporate insights into customer behavior by exploiting various sources of behavioral data, including trajectories and product interactions. Motivated by the complex and dynamic nature of modern stores, our research seeks to bridge the gap in retail inventory management. Our unique contribution lies in the development of a novel path planning method for robots, specifically tailored for an automated inventory management system. By focusing on customer trajectories and product interactions, we aim to enhance the arrangement and positioning of products within retail spaces. Our research is motivated by the need to address the challenges faced by retailers in optimizing store layouts and product placements. The proposed strategy utilizes a heatmap and a vision-based system to analyze spatial and temporal patterns of shopper behavior. This information is then employed to optimize robot navigation in both highly and less-visited areas. Trajectories and product interactions data from real store installations were utilized in simulation, providing valuable insights into optimal planning for mobile robots to visit Points of Interest (PoI). The active shopping cart tracking system generated heatmaps, while a vision-based system collected shopper-products interactions data. Subsequently, our approach was deployed on a real retail robot for inventory management, and the path planning source code was released. Our findings demonstrate that the path planned by our approach not only avoids collisions with static store sections but also optimizes paths in areas with significant customer-shelf activity.
– Shelf space is one of the most important tools for attracting customers’ attention in a retail store. This paper aims to develop a practical shelf space allocation model with visible vertical and ...horizontal categories. and formulate it in linear and non-linear forms.
– The research is mainly based on operational research. Simulation, mathematical optimization, and linear and nonlinear programming methods are mainly used. Special attention is given to the decision variables and constraints. Changing the dimensioning of the decision variables results in an improvement in the formulation of the problem, which in turn allows for obtaining an optimal solution.
– A comparison of the developed shelf space allocation models with visible vertical and horizontal categories in linear and nonlinear forms is presented. The computational experiments were performed with the help of CPLEX solver, which shows that the optimal solution of the linear problem formulation was obtained within a couple of seconds. However, a nonlinear form of this problem found the optimal solution only in 19 out of 45 instances. An increase in the time limits slightly improves the performance of the solutions of the nonlinear form.
– The main implication of research results for science is related to the possibility of determining an optimal solution to the initially formulated nonlinear shelf space allocation problem. The main implication for practice is to take into consideration the practical constraints based on customers’ requirements. The main limitations are the lack of storage conditions and holding time constraints.
– The main contribution is related to developing mathematical models that consider simultaneous categorization of products vertically, based on one characteristic, and horizontally, based on another characteristic. Contribution is also related to extending the shelf space allocation theory with the shelf space allocation problem model in relation to four sets of constraints: shelf constraints, product constraints, orientation constraints, and band constraints.
PDF
