A beginner's guide to tuning methods Montero, Elizabeth; Riff, María-Cristina; Neveu, Bertrand
Applied soft computing,
04/2014, Volume:
17
Journal Article
Peer reviewed
Display omitted
•We study the similarities and differences of well-known tuning methods.•We empirically compare their results when tuning numerical parameter of a standard genetic algorithm.•We ...establish guidelines for beginner's users of tuning algorithms.
Metaheuristic methods have been demonstrated to be efficient tools to solve hard optimization problems. Most metaheuristics define a set of parameters that must be tuned. A good setup of that parameter values can lead to take advantage of the metaheuristic capabilities to solve the problem at hand. Tuning strategies are step by step methods based on multiple runs of the metaheuristic algorithm. In this study we compare four automated tuning methods: F-Race, Revac, ParamILS and SPO. We evaluate the performance of each method using a standard genetic algorithm for continuous function optimization. We discuss about the requirements of each method, the resources used and quality of solutions found in different scenarios. Finally we establish some guidelines that can help to choose the more appropriate tuning procedure.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK
In this paper, we present a review of recent advances in wrapper feature selection techniques for attack detection and classification, applied in intrusion detection area. Due to the quantity of ...published papers in this area, it is difficult to ascertain the level of current research in wrapper feature selection techniques. Moreover, due to the wide variety of techniques and datasets, is difficult to identify relevant characteristics among them, regard it architecture, performance, advantages and issues. The reported results frequently are shown in heterogeneous way, as there are several metrics to measure the classification quality. From our review, we propose a classification taxonomy of the wrapper feature selection techniques in intrusion detection area, considering design, rationale, technical characteristics and common evaluation metrics. Also we consider a description of the common metrics and a brief discussion about the attack scenarios reported in this review. At the end of this work, we show the coverage of existing research, open challenges and new directions.
•A review of techniques for wrapper feature selection for classify attacks algorithms.•A classification taxonomy of wrapper feature selection techniques for IDS.•A start point to new researches, in wrapper feature selection perspective for IDS.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK, ZRSKP
Metaheuristics have been successfully applied to solve complex real-world problems in many application domains. Their performance strongly depends on the values of their parameters. Many tuning ...algorithms have already been proposed to find a set of suitable values. However, the amount of computational time required to obtain these values is usually high. Our objective is to propose a collaborative strategy to: (1) improve the quality of configurations obtained by tuner algorithms and (2) reduce the time consumed in the tuning process. Here, we introduce a novel opposite scoring (OS) strategy that learns from configurations that produce a positive and a negative effect in the target algorithm. However, OS guides its trajectory by choosing parameter configurations that decrease the performance of the target algorithm. For the learning process, OS stores the quality of all the evaluated configurations and computes a score for each value in the visited parameter configurations. Then, OS generates the initial set of configurations for a tuner, where values that obtain a better score will have a higher probability of being part of this set. We evaluate our proposal using the well-known Evolutionary Calibrator (Evoca). Also, we tune three different algorithms: an Ant Colony Optimization algorithm for solving the Multidimensional Knapsack Problem, a Genetic Algorithm for solving landscapes that follow the NK model (
N
components and degree
K
), and a Particle Swarm Optimization algorithm for solving continuous optimization problems. Results show that OS-Evoca obtains better quality configurations than Evoca, consuming less computational resources.
Full text
Available for:
DOBA, EMUNI, FIS, FZAB, GEOZS, GIS, IJS, IMTLJ, IZUM, KILJ, KISLJ, MFDPS, NLZOH, NUK, ODKLJ, OILJ, PILJ, PNG, SAZU, SBCE, SBJE, SBMB, SBNM, UILJ, UKNU, UL, UM, UPUK, VKSCE, ZAGLJ
Metaheuristics are powerful techniques for solving hard real-world problems in many application domains. Their behavior and performance strongly depend on their ability to efficiently explore and ...exploit the search space. A well-known metaheuristic is Ant Colony Optimization which has been successfully applied to solve many engineering problems. In this paper, we focus on ACO that solves Constraint Satisfaction Problems. In this context ACO has already shown to be able to solve many difficult CSPs, however, some problems are still very hard for this kind of technique. We introduce two strategies that allow improving ACO intensification and diversification process: one for learning in a pre-processing step and a second strategy to focus the search to a feasible space. Our results suggest that both the learning phase as well as the strategy to focus the search allow improving the ACO performance especially to solve hard CSPs. Moreover, these strategies can be applied to ACO for other application domains.
Display omitted
•We propose two strategies for ACO algorithms proposed for solving CSPs.•We use a strategy to define the initial pheromone matrix of ACO.•Another strategy to discriminate promising candidate instantiations.•Results show that both strategies allow to solve more hard problems.•We include an statistical analysis and fitness-distance plots.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
Recently, Opposition-Inspired Learning strategies were proposed to improve the search process of ant-based algorithms to solve combinatorial problems. In this paper, we propose a collaborative ...framework of these strategies called Multiple Opposite Synergic Strategy for Ants (MOSSA). Because of each strategy has a different goal, we expect that the ants algorithm will benefit from their collaboration. The algorithm strongly uses the pheromone matrix for accomplishing stigmergy. To evaluate our framework, we use a recently proposed algorithm to solve Constraint Satisfaction Problems named Focused Ant Solver. Results and statistical analysis show that using MOSSA, Focused Ant Solver is able to solve more problems from the transition phase.
•We propose an initialization strategy for ACO algorithms named MOSSA.•The strategy is a collaboration of three sub-colonies of ants.•Using our strategy Focused Ants Solver can solve more problems.•MOSSA-FAS and FAS are competitive to state-of-the-art ACO for CSPs.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
•Proposed algorithm to generate schedules for radiotherapy treatments.•Generated schedules take into account updated patients waiting time.•On-the-fly generated schedules reduce significantly the ...patients waiting time.
Scheduling Radiotherapy treatments for cancer patient is a major concern for hospital and clinics. The main problem consists in minimizing the patient waiting time in order to maximize the treatment effectiveness. Most of the modern scheduling approaches use expert systems based on scheduling heuristics and algorithms to develop detailed schedules, in order to efficiently map the patients requirements to the treatment capacity of the health center. In this paper, we propose RASON, a new heuristic based scheduling algorithm for radiotherapy treatments, which main objective is to minimize the average waiting time for each patient. In contrast to well-known existing approaches, our solution manages a priority list that can be dynamically updated according to both the patient category and his/her current waiting time. The generated schedule also impacts the minimization of the average tardiness of the first treatment sessions for each patient. We have evaluated our algorithm using both real data from the Institute of Radiotherapy in Santiago, Chilean and artificial cases generated with a self-developed generator called GeneRa. GeneRa is able to generate cases according to particular constraints inherent to several countries like UK, France and Italy. We show in our proposal evaluation that an on-the-fly scheduling has a great positive impact, allowing to reduce the average waiting time and tardiness for all patients categories. Our algorithm outperforms the JIT and ASAP well-known approaches, with a 95% statistical significance. Our scheduling algorithm allows to significantly reduce the treatment waiting time for different categories of patients. This is a major improvement for the patients as time and delays are crucial parameters to achieve the best effectiveness in cancer treatments.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK, ZRSKP
Parameter setting problem has demonstrated being a relevant problem related to the use of metaheuristics. ParamILS and I-Race are sophisticated tuning methods that can provide valuable information ...for designers as well as manage conditional parameters. However, the quality of parameter configurations they can find strongly depends on a proper definition of parameter search space. Evoca is a recently proposed tuner which has demonstrated being less sensitive to the setup of parameters search space. In this paper, we propose an effective collaborative approach that combines Evoca and I-Race as well as Evoca and ParamILS. In both collaborative strategies, Evoca is used to define a proper parameter search space for each tuner. Results demonstrated that the collaborative approaches studied are able to find good parameter configurations reducing the effort required to properly define the parameter search space.
Full text
Available for:
EMUNI, FIS, FZAB, GEOZS, GIS, IJS, IMTLJ, KILJ, KISLJ, MFDPS, NLZOH, NUK, OBVAL, OILJ, PNG, SAZU, SBCE, SBJE, SBMB, SBNM, UKNU, UL, UM, UPUK, VKSCE, ZAGLJ
ParamILS, I-Race and Evoca are well-known tuning methods designed to search quality parameter calibrations for metaheuristic algorithms. The set-up of parameter search space can strongly affect the ...performance of tuning methods. In this work we study how the parameter search definitions affect the quality of parameter calibrations delivered by these tuners. An experimental evaluation using two well known metaheuristic algorithms and a real life case is presented. We also provide some guidelines to consider when defining parameters search spaces according to the tuner used in order to obtain the best performance they can find.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK, ZRSKP
We present a hyper-heuristic approach to solve Orienteering Problem with Hotel Selection (OPHS). In practical applications, OPHS appears when a tourist is planning to visit various attractions and ...there is not enough time to reach all of them in a single day. Therefore, the tourist must build a tour within several days by selecting hotels, where each day has a different time budget. We propose a hyper-heuristic based on a Large Neighborhood Search, composed by a set of low-level heuristics that satisfy the different constraints associated with the problem. We put special emphasis on collaboration between low-level heuristics in order to guide the algorithm to more promising areas. We use 395 benchmark instances with known optimal solutions. This approach proves to be a more general method, with a simpler design compared to the literature, and is able to find 217 of the 395 known optimal solutions, in acceptable computational times.
Optimizing the usage of resources is an important topic in the development of technologies and computational services. The Google Machine Reassignment Problem is an NP-hard problem that is related to ...this crucial situation, based on the assignation of a set of processes into a set of machines trying to reduce several costs. This problem was proposed for the 2012 ROADEF/EURO challenge and since its introduction, many approaches have been proposed in order to reach better quality solutions or improve the execution time of the existing techniques. In this work, we review a significant number of recently proposed approaches. Due to the number of published papers, it is difficult to ascertain the level of current research in this area. In order to provide a useful guide to new interested researchers, we include up-to-date best-known results for benchmark instances, an analysis of the design of each technique and details of the experimental setup. We also present a classification and a taxonomy of the reviewed approaches based on the design of these techniques, considering their main components and the structure of the search strategies.
PDF