Comparability of RF-based Indoor Localization Solutions in Heterogeneous Environments: An Experimental Study

1
Comparability of RF-based Indoor Localization
Solutions in Heterogeneous Environments: An
Experimental Study
Tom Van Haute*, Eli De Poorter and Ingrid
Moerman
Department of Information Technology (INTEC)
Ghent University - iMinds
Gaston Crommenlaan 8, 9050 Ghent, Belgium
+32 9 331 49 46
{tom.vanhaute, eli.depoorter, ingrid.moerman}@intec.ugent.be
* Corresponding author
Filip Lemic, Vlado Handziski and Adam Wolisz
Telecommunication Networks Group (TKN)
Technische Universität Berlin (TUB)
Berlin, Germany
+49 30 314 23818
{lemic, handziski, wolisz}@tkn.tu-berlin.de
Niklas Wirström and Thiemo Voigt
SICS Swedish ICT
Stockholm, Sweden
+46 8 633 15 38
{niwi, thiemo}@sics.se
Abstract: The growing popularity of indoor localization research has resulted in a
significant amount of research papers describing and evaluating innovative localization
solutions. Unfortunately, the results from most of these research papers can not easily
be compared since they are evaluated in different environments, use different evaluation
criteria and typically tailor their solutions towards a single testbed environment. To
evaluate how these different conditions influence the localization performance, in this
paper an exhaustive set of experiments has been performed in which three different
localization solutions have been evaluated using multiple metrics in three different test
environments: two types of office environments and an industry-like factory environment.
None of the used localization solutions was previously optimized for any of these
test environments and they were all evaluated under similar conditions, e.g. similar
interference background and using the same evaluation points. The results reveal several
weaknesses in the evaluation methods used in the majority of existing scientific literature
of indoor localization solutions. More specifically, it is shown that (i) papers that use selfselected evaluation points can significantly influence their reported accuracy by artificially
selecting specific evaluation points that outperform other locations, (ii) the reported
accuracy can decrease significantly, up to a factor 10, when evaluating in an environment
for which the localization solution is not specifically tweaked and (iii) many inherent
trade-offs between different metrics, such as accuracy, energy consumption and response
delay, are hidden by reporting only on the accuracy of the solutions.
Keywords: Indoor localization; experimental comparison; benchmarking methodology;
performance metrics; fingerprinting; Time of Arrival, RSSI-based localization; IEEE
802.11; IEEE 802.15.4; w-iLab.t testbed; TWIST testbed; particle filter; Spray
2
T. Van Haute et al.
1 Introduction
• Four evaluation metrics (point accuracy, room
accuracy, energy consumption, response time),
1.1 Why Indoor Localization
• Three different test environments: two office
environments and an industrial-like open environment.
To the best of our knowledge, we are the first to evaluate
multiple localization solutions (and not only different
parameterizations of the same localization solution class)
in multiple environments using the same evaluation
procedures.
The remainder of this paper is organized as follows.
Section 2 discusses related work, including ongoing
efforts to standardize the evaluation of indoor
localization solutions. Next, Section 3 describes the
evaluated localization solutions: (i) a ToA-based IEEE
802.15.4 solution, (ii) a fingerprinting-based IEEE 802.11
solution and (iii) an IEEE 802.15.4 RSSI-based solution.
Section 4 discusses the used evaluation methodology
and evaluation metrics. Section 5 gives an overview
of characteristics of the used experimentation testbeds.
Afterwards, the localization solutions are evaluated and
the performance results of the solutions are compared
and discussed in Section 6 for office environment with
brick walls (TWIST), in Section 7 for office environment
with plywood walls (w-iLab.t I) and in Section 8 for open
industrial like environment (w-iLab.t II). This is followed
by a general overview in Section 9 where several lessons
learned are discussed. Finally, Section 10 concludes the
paper.
The emergence of satellite navigation systems - mainly
GPS [1] - has resulted in a significant increase
of personalized location-based services suitable for
guidance, navigation, tracking, recreation, security, etc.
However, the use of GPS is limited to outdoor
environments, whereas many commercial applications
are envisioned in indoor environments. Location-based
services are envisioned in many different indoor
environments: hospitals, airports, underground mines,
detention houses, etc.
A significant amount of work is available in scientific
literature describing and evaluating innovative
techniques or solutions for localization inside buildings.
As a result a wide range of indoor localization solutions
has been proposed using a variety of different RF
technologies (such as WiFi, RF, Bluetooth, 60GHz,
etc.) and non-RF technologies (such as infrared and
ultrasonic). However, a major problem is the lack of
comparability between indoor localization solutions.
• The majority of evaluations of indoor localization
solutions [2, 3] focus mainly on the accuracy of the
results whilst ignoring crucial application-level metrics
such as scalability, delay, energy consumption, cost,
simplicity, etc. Moreover, even the reported accuracy
is typically calculated using different calculation
statistics (average, median, percentiles, etc.), thereby
making comparison of solutions is almost impossible.
• In addition, even though each of the targeted
application domains has different environmental
characteristics, most of the existing solutions were
evaluated in one specific test environment. As a
result, it is impossible to gain insight in the
overall performance of these solutions under different
conditions.
Based on these observations, we argue that the current
state of the art is lacking comprehensive comparative
analysis of different localization approaches in multiple
deployment enthronements. The main reason for this
lack of comparability studies is the significant effort that
is currently required to perform localization experiments
in multiple experimentation facilities. The main goal
of this paper is to identify to what extent these
shortcomings influence the comparability of results in
existing scientific literature and to provide suggestions
for improvement. Therefore, we implemented three
typically used localization approaches and evaluated
their performance in multiple test environments
using the same evaluation methodology. Amongst the
evaluated solutions, we include:
• Two popular RF technologies (IEEE 802.11 and IEEE
802.15.4),
• Three localization approaches (ToA, fingerprinting,
weighted RSSI),
2 Related Work
Recently, there has been a growing awareness that
a more thorough way of comparing and evaluating
localization solutions is needed. This section gives an
overview of efforts related to evaluation procedures for
indoor localization solutions.
2.1 Evaluation Procedures for Indoor Localization
The need for a systematic and objective evaluation
methodology has been recognized by several authors [4].
Although no standardized methodologies are currently
available, several efforts are being made towards this
goal.
• The FP7 EVARILOS project a
[5] focuses on
the EVAluation of RF-based Indoor LOcalization
Solutions. The project published a first draft of a
benchmarking handbook [6, 7], describing methods
to calculate metrics, descriptive methods to describe
evaluation environments and methods for deciding
which evaluation points to use. The project is
also the first to point out that current scientific
literature lacks studies on the effect of interference
on indoor localization solutions, although interference
a The results described in this paper originate from the project
and have first been described in the public EVARILOS deliverable
D2.2 “Report on experiments without interference” [8].
Comparability of RF-based Indoor Localization Solutions in Heterogeneous Environments
is expected to be present at most sites where these
systems are installed b .
• In parallel, ISO (the International Organization
for Standardization) and IEC (the International
Electrotechnical Commission) have established a joint
technical committee, ISO/IEC JTC 1, to work on a
ISO/IEC 18305 standard on “Test and evaluation of
localization and tracking systems” [10] c . The draft
of the standard is not yet publicly available at the
time of writing of this paper, but it currently includes
a taxonomy of localization solutions and describes a
wide range of evaluation scenarios and performance
metrics. In contrast to the EVARILOS project, which
mainly focuses on RF-based localization solutions,
the ISO standard draft also considers other indoor
localization solutions that use a wide range of
input sensors such as inertial sensors, ultra-sound
sensors, etc. In terms of the evaluation approach,
the ISO standard focuses on evaluation of fixed set
of metrics and specifies a concrete enumerated set
of evaluation scenarios under which the solutions
should be evaluated. The EVARILOS benchmarking
methodology, in comparison, is more broad and
defines a basic ”vocabulary” for expressing different
evaluation scenarios, instead of constraining to a set
of few particular ”instances”. It also goes beyond
evaluation of simple performance metrics, and defines
a subsequent phase in which they can be translated
into use-case specific scores.
3
additional functional metrics, such as response delays,
and non-functional (deployment) metrics such as setup
time and required infrastructure.
The full list of potential metrics from these sources
is very large, especially since many of these metrics
can be calculated using multiple statistics (percentiles,
averages, median, distributions, etc.). Some metrics
are important mainly from a theoretical point of
view and as such are well-suited for analyzing and
improving algorithms of researchers [12], whereas other
focus on the performance of end-systems and as such
are more important for the industry. Unfortunately,
although the above sources strongly emphasize the
need for utilizing multiple criteria for evaluating indoor
localization solutions, none of these sources mention
which of the metrics are considered most important for
different application domains, nor do they offer insight
on the relation between different metrics (e.g. inherent
trade-offs).
Therefore, in the evaluation section of this paper, we
have included four functional metrics: point accuracy,
room accuracy, response delay and energy consumption.
These are the performance metrics from the EVARILOS
handbook [7], which can be found in Figure 1.
2.2 Evaluation Metrics for Indoor Localization
In more recent surveys, the importance of multiple
metrics becomes visible.
• Hui Lui et al. states in [4] that comprehensive
performance comparison requires not only accuracy,
but also needs to include precision, complexity,
scalability, robustness and cost.
• In the EvAAL project (“Evaluating AAL Systems
through Competitive Benchmarking”) [11], a
competition is held that aims at establishing
benchmarks and evaluation metrics for comparing
Ambient Assisted Living solutions. For this
competition, besides accuracy, also usability metrics
are defined such as installation complexity, user
acceptance, availability and interoperability with
AAL systems [11].
• A significant number of additional metrics can be
found in the aforementioned EVARILOS handbook [7]
and ISO/IEC 18305 draft [10], both including
b The outcome of initial studies on the influence of interference
on the localization solutions evaluated in this paper can be found
in EVARILOS deliverable D2.3 “Report on experiments with
interference” [9].
c ISO/IEC 18305 is being prepared by Joint Technical
Committee
ISO/IEC
JTC
1,
Information
technology,
Subcommittee SC 31, Automatic identification and data capture
techniques, Working Group 5, Real time locating systems. The
committee is currently referred to as ISO/IEC JTC1/SC31/WG5.
Figure 1: The EVARILOS metrics: a graphical
overview
2.3 Evaluation Environments for Indoor
Localization
It is a well-known fact that environmental conditions
significantly influence propagation characteristics.
Table 1 gives an overview of a number of recent research
papers evaluating localization solutions and describes
the environments they have been evaluated in.
It is clear from Table 1 that most existing indoor
localization solutions have been evaluated in office
environments, since these are the buildings which
are most readily available for researchers. Due to
the time-consuming nature of performing localization
experiments, most localization solutions are evaluated
only in a single environment. However, as will be shown
in Section 5, office environments can have very different
characteristics. Based on existing literature, it is not
clear how these differences in environment influence
the reported accuracy results. Therefore, this paper
will analyze the performance of multiple localization
solutions in three different environments: an office
environment with brick walls, an office environment
with plywooden walls and an industrial-like open
environment.
4
T. Van Haute et al.
Table 1
Overview of a few existing indoor localization solutions with the related environment, testbed and metrics
Solution
Energy efficient solution [12]
GSM fingerprinting [13]
WiFi Bayesian [14]
EZ localization [15]
Smartphone localization [16]
WiFi in tunnel [17]
Fingerprinting [18]
UWB fingerprinting [19]
Environment
Office
Office / home
Office
Office
University
Mining
University
Office / testroom
Used testbed
building on campus
university, research lab, house
Their own hallway
Office floor, Call Center
Berkeley campus
Tunnel in Guangzhou MTR
Fourth floor of university building
Anechoic chamber, office floor
2.4 Evaluation Points for Indoor Localization
In terms of which points to use in an environment
to evaluate the performance of a localization solution,
two main approaches are possible. For industry-related
testing, an evaluation track can be created that mimics
typical operations in a building. For example, the path of
a person can be recreated and only evaluation points on
this path can be used [12]. For more generic, applicationindependent testing, ideally the evaluation points should
be randomly chosen. Unfortunately, most research
papers manually select a number of evaluation points
based on subjective criteria such as accessibility. As
will be shown in Section 6, the accuracy of localization
solutions can strongly depend on the used evaluation
points, e.g. points near a wall versus open spaces. As
a result, the performance of localization solutions can
artificially be “improved” by selecting mostly evaluation
points which perform well for the evaluated solution.
As such, it is clear that future evaluations of indoor
localization solutions should use standardized evaluation
methods. To remedy this, future benchmarking
methodologies such as EVARILOS and ISO/IEC 18305
are creating standardized methods for generating
evaluation points. For this paper, all evaluated
localization solutions use the same evaluation points in
each testbed.
Used metrics
Point acc. & energy cons.
Point accuracy
Point accuracy
Point accuracy
Point accuracy
Point accuracy
Point accuracy
# of multipath components
3.1 Particle Filter using ToA and RSSI
Measurements
The first solution is designed by N. Wirström et al. [22].
The basic concept behind this localization solution is
the following: measurements are performed by letting
a stationary node transmit packets to the anchors that
reply with a hardware ACK (acknowledgment). The
initiating node measures both the time between the
transmission of the packet and the reception of the ACK,
and stores the RSSI values associated with the ACK.
These measurements are then processed using Spray [23],
a particle filter based platform.
The basic idea of the ToF ranging is to estimate
the distance between two nodes by measuring the
propagation time that is linearly correlated to the
distance between the nodes when they are in LoS.
Two-way ToF ranging, as opposed to one-way, does
not require tight time synchronization between sender
and receiver. This is an advantage since tight time
synchronization is hard to achieve in wireless sensor
networks (WSNs) [20].
The distance between nodes can be calculated according
to Equation 1 where c is the speed of light, tT oF is the
round-trip-time measurements, and tof f is an offset time
accounting for all processing delays in the system. This
includes the time for the sender to transmit the packet,
the time the receiver needs to process it, and send the
acknowledgment.
3 Evaluated Localization Solutions
To evaluate how different test environments influence
typical localization solutions, we selected three
localization solutions that use different wireless
technologies and that use different processing approaches
for estimating positions. The following localization
solutions were selected and implemented.
• An IEEE 802.15.4 based time-of-arrival solution.
• An IEEE 802.11 based fingerprinting solution.
• An IEEE 802.15.4 based RSSI triangulation solution.
Although more accurate solutions exist, these solutions
represent the most popular RF-based technologies
described in literature.
d=
c
(tT oF − tof f )
2
(1)
n
cycles
The measurements tT oF are computed as tT oF = ftimer
,
were ncycles is the number of measured clock ticks,
and ftimer is the frequency of the radio’s internal
crystal oscillator. In this case ftimer = 12 MHz. A single
measurement is not sufficient, however. The resolution
of a single clock allows for a spatial precision equal to
c
. For a 12 MHz clock, the resulting spatial
∆d = 2ftimer
resolution is 12.5 m. To achieve higher resolution, one
can average over a series of measurements, as proposed
by Mazomenos et al. [21]. This way, sub-clock precision
can be achieved.
Comparability of RF-based Indoor Localization Solutions in Heterogeneous Environments
5
3.1.1 Range Computation Methods
3.1.2 Using Spray to Estimate Location
Once the range measurements are collected, they have
to be transformed into actual distance measurements.
For this, a wide range of computation methods are
available. We have applied five different methods to the
measurements. Four of these use ToF measurements as
input, and one use RSSI measurements. The following
subsections describe the methods.
Once the raw range measurements are transformed to
distance estimations, the final location estimations are
obtained from Spray, a particle filter based localization
system that can be used to fuse multiple types of
measurements simultaneously.
In this case, i.e. using a single range based modality,
Spray generates particles that have both a position and a
weight, in a ring-shaped cloud (an annulus) around each
testbed node that has an associated range measurement
to the node that is to be localized. The distance
between each particle and its associated testbed node, is
determined by the sum of the range measurement and a
zero-mean normally distributed random variable with a
given variance.
Each particle is then evaluated using measurements from
all the other testbed nodes, on the basis of how the
particle’s position fits their measurements. This is done
by assigning a weight between 0 and 1 to the particle.
The more coherent the particle’s position is with the
measurement, the higher the weight. A final weight
for each particle is then computed by multiplying the
weights assigned in the evaluation phase.
Mazo: This model builds directly on Equation 1.
This is the model used by Mazomenos et al. [21].
The calibration step consists of estimating the constant
offset tof f by averaging over various ToF measurements
according to equation 2.
t̂of f =
N
2di
1 X
tT oF,i −
N i=1
c
(2)
k-sigma: This method was proposed by Pettinato
et al. [22]. It uses the variance between measurements
taken on different channels to improve range estimations.
The idea is that when two nodes are in line-of-sight, most
packets will travel the shortest path between the nodes,
regardless of the channel being used.
If the two nodes are not in the LoS, however, the different
frequencies of the different channels will cause slightly
different propagation paths, and result in different
ToF measurement values. The concept is captured in
Equation 3, where σ is the inter-channel standard
deviation. Calibration consists of estimating tof f and k
using linear regression.
d=
tT oF
− tof f − kσ
2
(3)
Least Squares: For this method, the calibration phase
consists simply of fitting data to the Equation 4, where a
and b are estimated using linear regression. This method
is model-free in the sense that it does not rely on a
physical model.
d = a + btT oF
(4)
Free Space RSSI: This method uses the free space
propagation model in the Equation 5, to transform RSSI
measurements to range estimations. In the equation,
Pr and Pt are the received and transmitted power,
respectively. Gr and Gt are the receivers and the
transmitters antenna gains, respectively. λ is the
wavelength and L is called the system loss factor.
Pr =
Pt Gt Gr λ2
(4π)2 d2 L
(5)
However, instead of determine these constants
individually we combine them into on single constant
K as in Equation 6, and estimate K using least squares
approximation.
Pr = K
1
d2
(6)
3.2 Fingerprinting Based Localization using WiFi
Beacon Packets RSSI Measurements
Another solution is provided by F. Lemic [24, 25] which
is based on WiFi fingerprinting. Fingerprinting methods
in the indoor localization are generally divided in two
phases. The first phase is called the training or offline
phase. In this phase, the localization area is divided
in a certain number of cells. Each cell is scanned a
certain number of times for different signal properties,
and using a methodology for processing the received
data a fingerprint of each cell is created. By using the
obtained training fingerprints the training database is
created and stored on the localization server. In the
second phase, known as the runtime or online phase, a
number of scans of the environment are created using
the user’s device. From the scanned data, using the
same predefined data processing method, the runtime
fingerprint is created and sent to the localization server.
At the server’s side the runtime fingerprint is compared
with the training dataset using the matching method.
The training fingerprint with the most similarities to
the runtime fingerprint is reported as the estimated
position. In the section below a general notion of the
WiFi fingerprinting is given using beacon packets RSSI
values.
Let Kt and M be respectively the number of WiFi APs
used for a localization procedure and the number of
training points in a given localization area. Furthermore,
let Nt be the number of scans of the area taken at a
training point m (m ∈ 1, ..., M ). During each scan the
vector of RSSI measurements from each visible AP used
for localization is collected. This vector has at most Kt
elements, but it is possible that it will have less elements
6
T. Van Haute et al.
if the user’s device is not in the range of a number of APs
or because beacon packets are lost due to interference.
After collecting Nt measurement vectors from different
APs at training point i the training matrix Sit is created.
t
The matrix Sit has Kt rows and Nt columns (SK
).
t ×Nt
The matrix of the training measurements from each
training cell is preprocessed training data. Based on
the method that each localization algorithm uses for
creating the fingerprint, from the matrices S t M training
fingerprints are created.
A similar procedure, with different parameters, is used
for creating the runtime scan of the RSSI measurements.
Let Kr be the number of WiFi AP used in the
localization procedure and visible to the user’s device at
a given location. The number of measurements taken by
the user’s device is equal to Nr . A runtime fingerprint
r
is a matrix of RSSI values SK
. A fingerprint is
r ×Nr
created using a method defined in the fingerprinting
based localization algorithm.
The principle of fingerprint based localization algorithms
is to accurately detect the similarities between training
dataset and runtime fingerprints. Due to the time and
energy constrains of a (usually wireless) user’s device,
the number of measurements in the runtime fingerprint
Nr is usually smaller than the number of measurements
taken while collecting training fingerprints Nt . For this
reason, the number of measurements given as an input
to a localization algorithm is equal to Nr . Furthermore,
only a subset of RSSI measurements from the APs that
are common to both training and runtime fingerprint is
given to the second phase of the localization algorithm.
For the evaluation, we use three fingerprint based indoor
localization algorithms which have been proposed in
previous research work.
ED of Averaged RSSI Vectors: The Euclidean
Distance (ED) of the averaged RSSI vectors is one
of the most basic and well known algorithms used
for fingerprint-based indoor localization algorithms [26].
The input to the matching method is an average value
of RSSI measurements obtained from each AP used
for localization in both training and runtime phase,
where Kr,t is the length of the vector. Let µt,m =
[RSSI t,1 , ..., RSSI t,k , ..., RSSI t,Kr,t ] be the vector of
averaged RSSI values from each AP obtained during
the training phase at cell m ∈ 1, ..., Mt , i.e. the
training fingerprint. In the same manner, let µr =
[RSSI r,1 , ..., RSSI r,k , ..., RSSI r,Kr ] be the vector of
averaged RSSI values from each AP obtained during the
runtime phase, i.e. the runtime fingerprint. The distance
between the training fingerprint at the cell m and the
runtime fingerprint is given as:
DE (µt,m , µr ) = |µt,i − µr,i |
(7)
The distance DEU (µt,m , µr ) is the ED distance between
the vectors of averaged RSSI values of the cell m and
runtime point. The cell with the smallest distance (also
called smallest weight) is reported as the estimated
position.
KL Distance of MvG Distributions of RSSIs:
The second fingerprinting based indoor localization
algorithm uses the Kullback-Leibler (KL) distance
between the Multivariate Gaussian distributions of RSSI
measurements from each AP used in the localization
procedure [26]. The algorithm assumes that the RSSI
values from each AP are distributed according to the
Multivariate Gaussian distribution. In other words, the
distribution of the RSSI values from each AP at one
cell can be written as N (µ, Σ). In the same manner as
in the previously presented algorithm, let µt,m and µr
be the vectors of the averaged RSSI values from each
AP in training phase at the cell m and in the running
phase, respectively. Furthermore, let the Σt,m and Σr
be the covariance matrices of the RSSI measurements
at training cell m and running point respectively. The
Multivariate Gaussian distributions of the training point
m and running point can then be written as Nt,m =
N (µt,m , Σt,m ) and Nr = N (µr , Σr ) respectively.
1
DKL (Nt,m , Nr ) = ((µSi,T − µSR )T (ΣSi,T )−1 (µSi,T − µSR )
2
+ tr(ΣSi,T (ΣSi,T )−1 − I) − ln|ΣSR (ΣSi,T )−1 |) (8)
where tr(·) denotes the trace of a matrix (sum of its
diagonal elements) and I is the identity matrix. The
matching method reports the cell with the smallest KL
distance as the estimated position.
PH Distance of RSSI Quantiles: Finally, as the
third fingerprinting method, we propose a new approach
using quantiles of the RSSI values from each AP for
creating fingerprints and the Pompeiu-Hausdorff (PH)
distance for estimating the similarities between the
training and runtime fingerprints. Using the quantiles
for indoor localization purposes is frequently used in
robotics, where robots are using quantiles of images of
the environments in order to localize themselves [27].
PH distance is usually used in image processing for
pattern recognition and measuring the dissimilarities
between shapes. As far as we know, using a combination
of quantiles of RSSI distributions and PH distance
for location estimation has not been proposed and
examined in literature. We find this approach promising
because a higher amount of information is provided to
the matching method. In other words, in our opinion
using only the vector of averaged RSSI values and
the covariance between measurements between different
APs may not be sufficient for precise localization. In
our case the q-quantile of the RSSI measurements
from each AP is calculated in two steps. The first
one computes the Cumulative Distribution Functions
(CDFs) of the RSSI measurements from each AP. The
second step calculates the quantiles, i.e. the RSSI values
with probabilities k/(q − 1), where k = 0, 1, ..., q − 1.
The result of the quantile calculation in both training
and runtime phase is a quantile matrix QK,q , where K is
the number of APs visible at the given location and q is
a number of quantiles. The similarities between the RSSI
quantiles from the training fingerprints and the runtime
Comparability of RF-based Indoor Localization Solutions in Heterogeneous Environments
7
fingerprint are computed using the PH distance metric.
The PH distance between two sets of quantiles is given
as follows:
DP H (Q1 , Q2 ) = max ( min (d(q1,k , q2,k )))
(9)
q1,k ∈Q1 q2,k ∈Q2
where d(q1,k , q2,k ) is the Euclidean distance (ED)
measurement. The training cell with the smallest PH
distance is reported as an estimated location.
3.3 Hybrid Model: Proximity & Weighted RSSI
A final localization solution [28] that has been
implemented and evaluated is a hybrid combination of
a range-based and a range-free algorithm. It includes a
range-based location estimator based on weighted RSSI
values. The main idea of RSSI is that the transmission
power PT directly affects the received power PR of a
signal. Using the Friis transmission equation, the linear
relationship can be stated as follows.
PR = PT ∗ GT ∗ GR
λ
4πd
2
(10)
In the equation GT , GR are the gains of transmitter and
receiver, respectively. λ is the wavelength of the signal
and d is the distance between sender and receiver. The
RSSI can be defined as the ratio of the received power
to the reference power PRef .
RSSI = 10 ∗ log
PR
PRef
(11)
Each RSSI value can be matched with a certain distance.
The proposed algorithm in [28] not only uses the RSSI
values to measure the distance between a fixed and
mobile node, but also the distance between the fixed
nodes. These values function as weight factors for the
distance calculation between the fixed and mobile node.
These weight factors are shown in Figure 2 as w12 , w13
and w23 . The distance from M to, for example, B1 can
be calculated as follows:
Distance(M, B1 ) =
RSSI(M, B1 ) ∗ w12 + RSSI(M, B1 ) ∗ w13
2
(12)
whereby wij :
wij =
Dist(Bi , Bj )
RSSI(Bi , Bj )
(13)
Previous results prove that these weight factors add
value to the accuracy. A drawback of the RSSI technique
is that these measurements are very sensitive to the
environment and any changes in it. The relationship
between the distance and RSSI is room dependent. For
example, signals in a long corridor propagate much
further because they reverberate through the long walls.
Figure 2: Weighted algorithm: schema
In contrast to the technique above, range-free algorithms
do not take RSSI-values into account. If a mobile sensor
node has a range of 10 meters, then a fixed node can
only receive his messages if the mobile node is maximum
10 meters away. This is the only information that is
used to calculate the position of a mobile node. For this
approach, it is important that the transmission power is
well configured. If the power is too low, the mobile node
could be out of range between two anchors. On the other
hand, if the power is too high, too many fixed nodes
will receive the beacon and a wrong estimation could be
made.
The latter problem can be solved by using a centroid
algorithm. This is only useful if there is a set of fixed
nodes with an overlapping coverage area. The beacon of
the mobile node is received by multiple fixed nodes. In
order to determine the position, the centroid of all the
receiving fixed nodes is calculated:
xM =
yM =
Pk
xn
Pk k
yn
n=0
n=0
(14)
k
In theory, this algorithm would give a 100 % guarantee
that room-accuracy is possible. However, experiments
have shown that this is not always the case. If the walls
are small enough andor do not strongly attenuate the
signal, signals can go through and a fixed node in a
different room can receive up the beacon. To prevent
incorrect location estimation, extra logic can be added
to the algorithm.
The extra logic takes the form of additional
environmental metadata. Suppose we have the exact
coordinates of all the walls, doors and nodes inside
a building. Knowing that every beacon has an index
number, the direct path could be checked between the
two fixed nodes who received the consecutive beacons. If
the mobile node goes from one room to another, without
using a door, then the last beacon can be dismissed. For
example (Figure 3) when node A2 receives a beacon and
8
T. Van Haute et al.
the next beacon is received by node B2 . It is impossible
to move directly from A2 to B2 without passing nodes
A1 and B1 . So the message that was received by beacon
B2 will be rejected.
cleanly decouples between evaluating individual metrics
and calculation of a final score used for ranking.
As illustrated on Figure 4, after collecting a set of
measurements necessary for the calculation of the
individual metrics, the methodology envisions the use of
weighting factors and thresholding for the calculation of
the final ranking score, reflecting the different impact
of the individual metrics for the particular application
scenario of interest.
4.2 Used Metrics
Figure 3: Three neighboring offices
With this optimization room-accuracy can be
guaranteed. Still, this solution has the drawback that a
lot of fixed infrastructure sensor nodes are necessary to
retrieve good results. If the network is sparse distributed,
then the algorithm would not work properly.
Finally, for the evaluation of this solution, experiments
were performed using four different Tx power levels
(Tx3, Tx7, Tx19 and Tx31), as shown in Table 2.
Table 2
Used Tx power levels for the weighted RSSI
localization experiments
Tx power level
3
7
19
31
Output power [dBm]
-25
-15
-5
0
4 The Benchmarking Methodology
4.1 Introduction
One of the major problems of indoor localization
is the challenge of reproducing research results in
real life scenarios and the inability to compare their
performance due to evaluation under individual, not
comparable and not repeatable conditions. Therefore,
contrary to previous approaches, our benchmarking
approach does not focus exclusively on the accuracy of
the evaluated localization approach, but also considers
other performance measures that are relevant from the
point of view of practical deployment of localization
solutions such as energy efficiency and response time.
The metrics that will be used for the evaluation of the
solutions will have a critical impact on the final score.
A classical mistake by other comparison and evaluation
tools is only using the point accuracy as a reference for a
good or bad working solution. In this paper, we will take
others metrics into account as well, which are defined in
the EVARILOS Benchmarking Handbook [6].
4.3 Used Scenarios
Each solution is evaluated using a predefined scenario
in each testbed. These are based on the generic
scenario descriptions of the EVARILOS Benchmarking
Handbook. In the next paragraphs, we will describe each
scenario of each testbed. A detailed overview of each
testbed is given in Section 5.
4.3.1 TWIST Testbed
The scenario is instantiated on the 2nd floor of
the TWIST testbed, and can be characterized as a
“small office environment” according to the EVARILOS
Benchmarking Handbook. The evaluation points used to
evaluate the localization solutions are shown in Figure 5.
These points were selected based on the Latin Hypercube
principle, taking into account that there are limitations
due to unreachable places.
Figure 5: TWIST evaluation points utilized for the
first benchmarking scenario
Figure 4: Transform measurements to scores using
metrics
Due to variation in the sensibility of different use-case
scenarios on the individual metrics, the methodology
4.3.2 w-iLab.t I Testbed
At w-iLab.t I testbed, we use the third floor to execute
the experiments. On this floor, 57 nodes are available for
the experiments. An overview of the third floor is given
in Figure 12. There is no actual difference between the
Comparability of RF-based Indoor Localization Solutions in Heterogeneous Environments
Figure 6: Measurement points in the w-iLab.t I
testbed
green and the blue dots, it is for reservation purposes
only.
Unfortunately, not the whole floor can be considered as
a test area. Some private offices, technical staff room,
etc. are not available for measuring. The unreachable
zones are marked with a red layer. In order to define
the measurement points, we used a grid (see Figure 6).
The decision has to be taken without premeditation.
Therefore, a randomizer is used. To avoid measurement
points close to each other, making an unbalanced
distribution, the principle of the Latin Square is applied.
9
Berlin. The TWIST instance at the TKN office building
is one of the first and most popular remotely accessible
testbeds [29]. It has 204 SUT sockets, currently
populated with 102 eyesIFX and 102 Tmote Sky nodes
(Figure 8, with location of TMote Sky nodes indicated
with red and locations of WiFi access points with purple
dots.). Tmote Sky nodes use a Tiny OS development
environment. They consist of a TI MSP430 processor
running at 8 MHz, 10 KB of RAM, 1 Mbit of flash
memory and an IEEE 802.15.4 compliant Chipcon
CC2420 radio operating at 2.4 GHz with a maximum
indoor range of approximately 100 m. Each node
includes sensors for light, temperature, and humidity.
The hardware setup is extendibles with a large variety
of other radios (e.g Software Defined Radio, sensing
engine), as long as the radio has a USB or RS232
serial interfaces. The nodes are deployed in a 3D grid
spanning 3 floors of an office building at the TUB
campus, resulting in more than 1500 m2 of instrumented
office space. In small rooms (∼14 m2 ), two nodes of each
platform are deployed, while the larger ones (∼28 m2 )
have four nodes (Figure 9). This setup results in a fairly
regular grid deployment pattern with intra node distance
of 3 m. Within the rooms the sensor nodes are attached
to the ceiling.
4.3.3 w-iLab.t II Testbed
In this testbed, the 26 measurement points are well
spread over the area. These are selected by randomness
in each sub-area. This is shown in Figure 7. Special in
this setup is that there is no physical person present
in the building. Everything is controlled remotely using
robots. In this way, the repeatability of the measurement
point is very high. On the other hand, this “open
environment” is made of metal walls and contains a lot
of metal objects, making it very challenging for accurate
localization.
Figure 8: TWIST testbed: nodes
Figure 9: TWIST testbed: map (2nd floor)
Figure 7: The measurement points of w-iLab.t II
testbed
5 Test Environments
5.1 TWIST Testbed in Berlin
The TKN Wireless Indoor Sensor Network Testbed
(TWIST) is a multiplatform, hierarchical testbed
architecture developed at the Technische Universität
For specific purpose of benchmarking of RF-based indoor
localization, in addition to the described sensor network,
the TWIST infrastructure consists of multiple other
devices, as described in [30, 31]. Deployed WiFi access
points are commercial of-the-shelf TL-WDR4300 routers
(Figure 10). The WiFi routers can serve two functions.
They can be used as a part of the localization solution, if
particular solution requires WiFi anchor points. At the
same time, some routers can also be used for creating
different types and amounts of IEEE 802.11 traffic in
order to generate controlled WiFi interference.
10
T. Van Haute et al.
Figure 10: TWIST testbed: hardware components
For supporting mobility and automation of the
localization measurements multiple TWISTbot robotic
platforms (based on the TurtleBot design from Willow
Garage), are used. Their function is to carry nodes that
need to be localized through the measurement points and
report the ground truth position.
Furthermore,
the
TWIST
infrastructure
is
complemented by several WiSpy sensing devices: these
are low-cost spectrum scanners that monitor activity
in the 868 MHz, 2.4 and 5 GHz spectrum, and output
the measured RF energy and the quality of the received
signals. Also, for more precise sensing of the wireless
environment spectrum analyzers are used. Except for the
before mentioned WiFi routers and TWIST platform,
for generating interference Rohde & Schwarz signal
generator is used. Signal generator can be used for
generating arbitrary RF signals, and its usual usage is
for generating microwave interference.
5.2 w-iLab.t I Testbed at De Zuiderpoort
The w-iLab.t I testbed is located at “De Zuiderpoort”
in Ghent, Belgium. The infrastructure is distributed on
three floors (18 x 90m) of the iMinds office (Figure 12).
The network consists of 200 nodes. Every w-iLab.t node
is generic and is equipped with one or more sensor nodes,
an intermediate node with 2 WiFi 802.11 radios, the
environment emulator and a Bluetooth interface. As in
TSIT testbed, the sensor nodes are Tmote Sky motes.
The intermediate nodes (called iNodes, Figure 11) are
Alix 3C3 devices running Linux. These are mini PCs
equipped with Ethernet, USB, serial, VGA, audio and
two IEEE 802.11 a/b/g interfaces. All the iNodes
are connected to the management backbone using
Power-over-Ethernet switches, making it possible to
power up/down the iNodes as needed without physical
interaction with the iNodes. The iNodes can become an
active member of the experiment as it is possible to
adjust the kernel, the driver, to add click router code or
to add java-based applications.
Finally, the Environment Emulator (EE) is located in
between the iNode and the sensor node. Using the
EE, it is possible to emulate the behavior of any
type of sensor or actuator without the need for real
sensoractuator hardware or the development of a fullblown sensor application. It is possible to emulate the
battery depletion, depending on the real life power
consumption of the sensor node. When the node’s
battery is depleted or the node is destroyed (e.g., in
an explosion), the node can be switched off. The EE
can be programmed to emulate a sensor event (e.g.,
temperature rise, motion detection), an actuator event
or to support voice streams. Further, the EE can be used
to monitor the energy consumption of each individual
sensor. Altogether, this means that it is possible to
assess the complete usability of a certain wireless sensor
and actuator network application or protocol in a reallife environment. The initial core of w-iLab.t was based
on the widely used MoteLab testbed from Harvard
University. According to the EVARILOS Benchmarking
Methodology This building belongs to the category
“Plywooden walls” and the size is “Big”.
This is a classic office environment where multiple
devices communicate wirelessly with each other. Laptops
using WiFi and Bluetooth, smartphones using the 3G
network. Here we consider typical office applications like
email, file transfer, videoaudio conferencing and web
surfing. The office environment is a live environment.
Meaning the interference in this testbed is uncontrolled.
During daytime several people are working in these
buildings. So the w-iLab.t I is a testbed with very
realistic office interference. The cost of this realistic office
environment is the uncontrollable interference.
The w-iLab.t I testbed is centrally managed for
control and monitoring purposes. It supports easy
configuration and deployment, including installation of
new software, protocols and middleware components via
an intuitive web-based interface. Registered users can
upload executables, associate those executables with the
nodes (both sensor nodes and iNodes) to create a job,
and schedule the job to be run on w-iLab.t I. During the
job all messages and other data are logged to a database,
which is presented to the user upon job completion and
then can be used for processing and visualization.
All the possibilities of the complete testbed, the
environment emulator scenarios and events, a
visualization and a graphical analysis tool, are accessible
through a web interface. The visualization too can
visualize any type of node status andor link information
on a map of the building, while the graphical analyzer
plots out the data. The information for both tools is
gathered from the database through the use of user
customizable MySQL statements, making it extremely
flexible. External users can access the testbed over a
secured OpenVPN connection.
Figure 11: iNode mounted to the ceiling of the
w-iLab.t I wireless testbed
Comparability of RF-based Indoor Localization Solutions in Heterogeneous Environments
Table 3
Statistical information about the performance of
the particle filter algorithm in TWIST testbed
Metric
Average error [m]
Min. error [m]
Max. error [m]
Median error [m]
RMS error [m]
Room accuracy [%]
Response time [ms]
5.3 w-iLab.t II in Zwijnaarde
The w-iLab.t II testbed is located in “Zwijnaarde”,
above a cleanroom. At this location, there is (almost) no
interference. It is one open space where 60 fixed nodes are
distributed over an area of 70 x 25 m (Figure 13). In this
environment there are also 20 mobile nodes. These nodes
are based on a vacuum cleaning robot and are extended
with a radio for remote control and accurate positioning
algorithms (with rasters on the floor). Due to the fact
that the movement of these robots is controlled, mobility
is reproducible. The fixed nodes are marked with blue
spots while the mobile nodes have orange spots on the
map in Figure 13. Every node location contains (i) a
Zotac embedded PC, (ii) an environment emulator (see
w-iLab.t I), (iii) an iMinds Rmoni sensor node, (iv) a
Bluetooth dongle and some of them have a web-camera.
These nodes are remotely powered by Racktivity PDUs.
RSSI
ToA
4.35
0.62
12.99
3.22
5.28
45.00
14 285
5.56
0.68
22.47
3.91
7.11
30.00
14 282
Measurements are collected for over a minute at each
measurement point. Most of this time is spent trying
to reach nodes that are not within reach, and finding
the channel that a reachable testbed node currently is
using. The response time could be decreased significantly
by dedicating a single channel for communication to
be used before starting the ranging phase. Moreover,
range estimations do not improve significantly after 1300
measurements, as shown in Figure 14. Therefore we
use only the first 1300 collected values for our range
estimations. This also helps limiting the response time
because each measurement takes on average 4 ms. We
use the same approach for the RSSI measurements,
although the figure shows that 50 measurements are
likely to be enough. The figure also shows that after
approximately 500 measurements the ToA based method
performs better.
5.5
Abs. Range Error vs. Number of Measurements
rssi
toa
5.0
4.5
Abs. Error (m)
Figure 12: The w-iLab.t I wireless testbed: map
11
4.0
3.5
3.0
2.50
Figure 13: The w-iLab.t II wireless testbed: map
6 Results in TWIST Testbed
500
1000
1500
2000
# Measurements
2500
3000
Figure 14: The absolute range error for ToA decreases
with the number of measurements until approximately
1300 measurements. The RSSI error fluctuates about
the same value, and is not improved by additional
measurements
6.1 RSSI and ToA with Particle Filter
In this section we evaluate the particle filter localization
approach outlined in Section 3.1. Table 3 presents the
summarized results. In this table, “Min.” stands for
Minimum, “Max.” for Maximum and “RMS” for Root
Mean Square. These abbreviations are also used in the
other tables.
Figure 15 shows the CDFs for the absolute range
errors and the localization errors. The RSSI based
range estimation performs better than the ToA based
estimation, although Figure 14 shows that ToA should
give better results for a high number of measurements.
The reason for this is that only about 50% of the pairwise ranging procedures result in 500 measurements or
12
T. Van Haute et al.
more, and only about 10% result in 1300 measurements
or more.
1.0
Abs Range Error CDF (twist, reference_scenario)
Probability
0.8
0.6
0.4
are shown in Table 4. The results show that the PH
Distance of RSSI Quantiles give comparable results with
the ED Distance of Averaged RSSI Vectors in office
scenarios (see also Figure 16). The results also show that,
when more beacons are collected and thus the response
time increases, the PH has a slightly better overall
performance in terms of accuracy. This improvement
will be more emphasized in the open space scenario
(see results in Section 8). The minimum error of all
solutions equals zero, which is possible because some of
the fingerprints taken during the training set are at the
same locations that were used for the evaluation of the
algorithm.
Table 4
0.2
0.00
1.0
rssi
toa
5
15
10
Abs. Error (m)
20
25
Localization Error CDF (Twist, reference_scenario)
Probability
0.8
0.6
0.4
0.2
0.00
rssi
toa
5
10
Error (m)
15
20
25
Figure 15: CDFs for the absolute range error (top)
and the localization error (bottom)
The power consumption of both the target node and
the testbed nodes is approximately 105 mW. It is
computed as the mean of the transmission and reception
power consumptions. The energy consumption per node
is especially important when battery powered devices
will be used, since it directly impacts the lifetime
of a battery-powered localization solution. The node
energy consumption can also be used to calculate the
overall energy consumption. The infrastructure nodes
are always on, and a total of 68 testbed nodes are used.
As a result, the continuous total power consumption can
be calculated to be 7.1 W for the infrastructure. The
mobile node is only on during the response time, which
is in the order of 15 s, resulting in an average energy
consumption of 1.5 J per measurement.
6.2 Fingerprinting
This section evaluates the fingerprinting localization
approach described in Section 3.2. The accuracy results
Statistical information about the performance of
fingerprinting algorithms in TWIST testbed
Metric
Average error [m]
Min. error [m]
Max. error [m]
Median error [m]
RMS error [m]
Room accuracy [%]
Response time [s]
KL
ED
PH
2.77
0.00
5.71
2.98
3.39
50.00
35.67
2.16
0.00
6.35
2.48
2.95
80.00
35.11
2.02
0.00
6.35
2.52
2.79
85.00
35.12
Since a localization solution, in general, contains many
configurable parameters, we expect that they will
typically be offered to end users using predetermined
configuration setting. As such, it is important to be
aware of the inherent trade-offs that are made by the
developer of the solution. This is especially important
when considering also additional metrics such as the
response time. For this solution, the time during which
fingerprints are collected (e.g. the time needed before
a location estimate could be generated) was set to
35 seconds (excluding the off-line time required for
fingerprinting). Figure 17 and 18 show the trade-offs
between response and point and room level accuracies
for fingerprinting based solutions in TWIST testbed. As
more fingerprints are collected, a better match can be
made in order to better estimate the position. Lower
response times are possible, at the cost of decreased
accuracy. Especially more complex algorithms (such as
the PH distance) require more samples to estimate
the distributions of the RSSI values. As such, when
comparing different localization solutions, the targeted
response time has an important influence on the selection
of the best algorithm.
Finally, the energy consumption of the infrastructure
nodes (TPLINK 4300 router) is on average 0.5 W,
whereas the energy consumption of the used mobile
devices (MacBook Pro AirPort Extreme NIC) was on
average 7 W.
6.3 Proximity & Weighted RSSI
This section evaluates the RSSI based localization
approach described in Section 3.3. The obtained
accuracy is summarized in Table 5 for different
transmission powers. The average accuracy is relatively
Comparability of RF-based Indoor Localization Solutions in Heterogeneous Environments
13
low: the concrete walls in the building cause
unpredictable signal attenuation, resulting in less
accurate estimations of the true location. Using lower
transmission powers causes less signals to propagate to
multiple rooms, hence the better performance of low
transmission powers. A cumulative distribution function
of the errors is shown in Figure 19.
Table 5
Statistical information about the performance of
the hybrid algorithm in TWIST testbed
Metric
Average error [m]
Min. error [m]
Max. error [m]
Median error [m]
RMS error [m]
Figure 16: CDF of the localization error of
fingerprinting based solutions in TWIST testbed
Figure 17: Fingerprint collection delay versus point
accuracy
Figure 18: Fingerprint collection delay versus room
level accuracy
Room accuracy [%]
Response time [ms]
Tx3
Tx7
Tx19
Tx31
4.63
0.75
10.20
4.39
5.13
26.67
1503
7.08
0.83
17.52
6.81
7.75
6.70
1507
6.93
0.80
18.93
6.68
7.82
13.45
480
8.31
0.82
19.31
8.63
9.24
9.56
460
Figure 19: Distribution of the RMS localization error
in TWIST testbed
To estimate the position, the anchor points collect RSSI
values from the beacons transmitted by the mobile
node. All these RSSI values are collected and merged in
the position calculator. There, a translation from RSSI
values into coordinates is made. For low transmission
powers, the corresponding response delay is about 1.5 s
(exact values are given in Section 6.4), with an energy
consumption of about 31 mW for the mobile node.
To analyze the spatial distribution of the errors, a
box-plot of the accuracy per measurement point is
shown in Figure 20. The overall performance is for
each measurement point the same, there are no obvious
outliers. Noticeable, the worst minimum values are
obtained in the corridor (measurement points 283, 285
and 286) and in the room where no LoS nodes are
available (measurement point 240). If the results of these
rooms are excluded in the room accuracy calculation,
then the results are marginally better (e.g. for a Tx
power = 7 the room accuracy increases to 21.3 % instead
of to 10.3 %), but even in the rooms where the nodes
T. Van Haute et al.
Figure 20: Proximity & weighted RSSI solution spatial distribution of accuracy error in TWIST
testbed, including maximum error, minimum error,
quartile 1, quartile 2 and median error
were available, the average error distance of almost 5 m
is not enough to guarantee room accuracy: only 33.8 %
of all the measurement points are in the same room.
The main reason for these results is that proximity
requires extremely low transmission powers: even using
the lowest transmission powers from the TMoteSky
nodes, signals still easily penetrated the walls. Finally,
the box-plot of measurement point 220 is also
remarkable. The most logical explanation for this result
is that only a few fixed nodes received the beacons of the
mobile node. As a result, the calculator does not have
much data to process. This makes the result very stable,
but not necessarily more accurate.
6.4 Conclusions from the TWIST Experiments
An overview of the performance of the different
localization solutions is given in Table 6. In terms
of accuracy, the best performing solutions are the
fingerprinting localization techniques. Since the TWIST
building represents typical office buildings with concrete
andor brick walls, different rooms are very diverse in
terms of their wireless characteristics. Model based
localization solutions (such as RSSI based solutions)
suffer from degraded performance due to unexpected
obstacles. In contrast, localization solutions that exploit
this diversity, such as fingerprinting based approaches,
obtain the highest accuracy.
When also considering other metrics, these conclusions
need to be nuanced. In terms of response time,
fingerprinting performs worst, due to the need to collect
a minimal number of beacons. Since the beacon interval
is not always configurable on already deployed access
points, it is not always possible to decrease the response
time when using existing off-the-shelf access points. In
contrast, the ToA solutions can give location estimates
in only halve of the response time (about 15 s vs. 35 s),
and as shown earlier in Figure 14, the response time of
the RSSI based solutions can theoretically be reduced to
about 200 ms.
In terms of energy consumption, the devices used
in the fingerprinting solution consume most energy,
which means that battery-powered solutions will have
a low network lifetime when using IEEE 802.11
based fingerprinting. The energy consumption of the
Response time [ms]
Energy efficiency [mW]
TWIST testbed: summarized benchmarking
results
Average Error [m]
Table 6
Room accuracy [%]
14
Algorithm
Mobile Fixed
Particle filter solution
Spray RSSI
4.35
45.00
14 285
Spray ToA
5.56
30.00
14 282
Fingerprinting solution
KL Distance
2.7
50.0
∼ 35 000
ED Distance
2.2
80.0
∼ 35 000
PH Distance
2.0
85.0
∼ 35 000
Hybrid solution
TX Power = 3
4.6
26.7
1 503.1
TX Power = 7
7.1
6.7
1 507.6
TX Power = 19
7.9
13.4
480.6
TX Power = 31
8.7
9.5
460.9
∼ 105
∼ 105
∼ 105
∼ 105
∼ 7000 ∼ 500
∼ 7000 ∼ 500
∼ 7000 ∼ 500
∼ 30.9
∼ 35.1
∼ 47.1
∼ 57.6
∼ 47.4
∼ 47.4
∼ 47.4
∼ 47.4
IEEE 802.15.4 devices is significantly lower. However,
due to the large number of measurements required,
the ToA still consumes twice the energy of the hybrid
solution. It is clear that the energy consumption could be
further optimized, albeit at the cost of longer response
times.
Finally, the fingerprinting approach, although the most
accurate, has one other disadvantage which is not taken
into account by considering only the shown metrics.
More specifically, the need for an off-line training
phase and the need for retraining if the environmental
conditions change can significantly impact the accuracy
over time in realistic conditions. This clearly shows the
need for an objective comparison method that takes
into account multiple evaluation criteria when comparing
localization solutions.
7 Results in w-iLab.t I Testbed
7.1 RSSI and ToA with Particle Filter
The results for the particle filter based localization using
RSSI and ToA range measurements are presented in
Table 7. Measurements are collected and processed in
the same way as in the TWIST testbed.
Table 7
Accuracy of the particle filter in the w-iLab.t I
testbed
Metric
Average error [m]
Min. error [m]
Max. error [m]
Median error [m]
RMS error [m]
Room accuracy [%]
Response time [ms]
RSSI
ToA
7.79
3.59
14.04
7.09
8.43
30.00
55.45
7.16
1.51
14.31
6.09
7.92
20.00
55.44
Comparability of RF-based Indoor Localization Solutions in Heterogeneous Environments
12.5
Abs. Range Error vs. Number of Measurements
rssi
toa
12.0
Abs. Error (m)
11.5
11.0
10.5
10.0
9.5
9.00
500
1000
1500
2000
# Measurements
2500
3000
Figure 21: The absolute range error for ToA decreases
with the number of measurements until approximately
2500 measurements. The RSSI error fluctuates about
the same value, and is not improved by additional
measurements
1.0
Abs Range Error CDF (wilab1, reference_scenario)
Probability
0.8
0.4
0.2
1.0
Figure 21 shows how the number of measurement affects
the accuracy. As in TWIST testbed, ToA benefits from
more measurements, while RSSI based ranging does not.
A major difference to TWIST, also shown by the CDFs
to the left in Figure 22, is that both types of range
measurements have much larger errors. A reason for
this can be that more testbed nodes far away from
the measurement points are reachable, and that the far
traveling signals are subject to multi-path effects to a
greater extent, resulting in unpredictable attenuation
that is not captured by the free-space model. The
response time is also much higher than in TWIST
testbed. This is also because more testbed nodes are used
for each measurement. The response time can be reduced
if no measurements are collected after a certain number
of testbed nodes have been used.
The right graph in Figure 22 shows the CDF for the
localization error. The median error is about the same
as that for the range measurements in the left graph (8
and 7 m, respectively), but has lower errors above the
median.
The power consumption for the mobile node is the same
as in the TWIST experiments, i.e. 105 mW. Although
less testbed nodes are used in this experiment, we
consider the total infrastructure power consumption to
be of the same magnitude as in TWIST, i.e. 7 W.
7.2 Fingerprinting
0.6
0.00
rssi
toa
5
15
10
Abs. Error (m)
20
25
Localization Error CDF (WiLab1, reference_scenario)
This section evaluates the fingerprinting localization
approach described in Section 3.2 in the w-iLab.t I
testbed. The accuracy results are shown in Table 8. In
general, the accuracy is lower than in the TWIST testbed
environment. The CDF of localization error is shown in
Figure 23. The decrease in accuracy can be explained
by the fact that the w-iLab.t I testbed is an office
environment that uses plywood walls, which attenuate
the signals less than the concrete walls in the TWIST
testbed.
Table 8
0.8
Statistical information about the performance of
the fingerprinting algorithms in w-iLab.t I testbed
Probability
Metric
Average error [m]
Min. error [m]
Max. error [m]
Median error [m]
RMS error [m]
Room accuracy [%]
Response time [s]
0.6
0.4
0.2
0.00
15
rssi
toa
5
10
Error (m)
15
20
25
Figure 22: CDFs for the absolute range error (top)
and the localization error (bottom)
KL
ED
PH
6.15
1.12
15.86
4.37
7.25
50.0
24.98
2.37
0.00
5.50
2.75
3.34
80.0
24.16
2.75
0.00
11.0
2.75
3.76
85.0
24.36
As a result, the evaluated locations have less diversity in
terms of the received signal strengths, and are thus more
difficult to uniquely characterize in a fingerprint. This
effect will have an even greater influence in the results of
open space environment of w-iLab.t II (Section 8). The
point and room level accuracy vary with the performance
delay, as presented in Figure 24 and Figure 25.
16
T. Van Haute et al.
7.3 Proximity & Weighted RSSI
The location accuracy of the weighted RSSI based
localization solution in w-iLab.t I testbed is shown in
Table 9. The CDF of the localization error can be found
in Figure 26. Because the plywood walls do not attenuate
the signals significantly, locations need to be determined
based on weighted RSSI values (rather than proximity)
even when using low transmission powers. As a result,
in contrast to the experiments in the TWIST testbed,
where the localization accuracy depends strongly on the
transmission power, the results in w-iLab.t I testbed are
less dependent on the transmission power.
Table 9
Statistical information about the performance of
the hybrid algorithm in w-iLab.t I testbed
Metric
Figure 23: CDF of the localization error of the
fingerprinting based solutions in w-iLab.t I testbed
Average error [m]
Min. error [m]
Max. error [m]
Median error [m]
RMS error [m]
Room accuracy [%]
Response time [ms]
Tx3
Tx7
Tx19
Tx31
7.64
0.07
48.77
5.87
9.35
18.62
2100
8.86
0.04
65.98
7.63
10.15
16.27
113
7.47
0.17
45.23
6.18
8.83
12.60
108
8.21
0.04
45.23
7.21
9.44
9.46
110
Figure 24: Fingerprint collection delay versus point
accuracy
Figure 26: Distribution of the RMS localization error
in w-iLab.t I testbed
Figure 25: Fingerprint collection delay versus room
level accuracy
The spatial spread of the accuracy is shown in Figure 27
using a box-plot. The measured points more in the center
of the testbed have a higher accuracy then those at the
edges, because the edge evaluation points are outside the
grid of used anchor points. For example, in Figure 28
a clear bias in the estimated locations can be observed
caused by the fact that all anchor nodes are located at
the same side of the evaluation point. This highlights
the importance of using anchor nodes outside the area
that is evaluated, which is a requirement that is not
found for the fingerprinting solutions. Finally, Figure 29
shows the room accuracy of location estimation. It is
interesting to note that it is not possible to predict the
room accuracy based only on the point accuracy, because
Comparability of RF-based Indoor Localization Solutions in Heterogeneous Environments
the room accuracy depends strongly on random factors
such as the direction of the inaccuracies.
50,00
45,00
Error distance [m]
40,00
35,00
30,00
25,00
20,00
15,00
10,00
5,00
17
metal cupboards and metal ceilings. This demonstrates
that the performance of localization solutions in typical
environments is influenced by many factors besides the
building construction materials, and highlights the fact
that localization performances measured in a empty
building should not be considered representative for the
performance of said solutions when the buildings are
actively used.
0,00
4
5
6
7
8
9
10
Measurement points in w.iLab-t I
Table 10 w-iLab.t I testbed: summarized benchmarking
results
1600
1400
1200
1000
800
600
400
200
0
Algorithm
Mobile Fixed
Particle filter solution
Spray RSSI
Spray ToA
Measured points
True point
0
500
1000
1500
2000
2500
3000
80
70
60
50
40
30
20
10
0
7.79
7.16
3500
Figure 28: Biased spread of the location estimates
resulting from evaluating measurement locations
outside the grid of anchor nodes.
Room accuracy (%)
Average Error [m]
Figure 27: Proximity & weighted RSSI solution spatial distribution of accuracy error in w-iLab.t I,
including maximum error, minimum error, quartile 1,
quartile 2 and median error
Energy efficiency [mW]
3
Response time [ms]
2
Room accuracy [%]
1
TX 3
30.00
20.00
55 448
55 444
∼ 105 ∼ 105
∼ 105 ∼ 105
Fingerprinting solution
KL Distance
ED Distance
PH Distance
6.15
2.37
2.75
50.00
80.00
85.00
∼ 24000
∼ 24000
∼ 24000
∼ 7000 ∼ 500
∼ 7000 ∼ 500
∼ 7000 ∼ 500
Hybrid solution
TX
TX
TX
TX
Power
Power
Power
Power
=
=
=
=
3
7
19
31
7.64
8.86
7.47
8.21
18.62
16.27
12.60
9.46
2100.74
113.15
107.99
110.17
∼ 30.9
∼ 35.1
∼ 47.1
∼ 57.6
∼ 47.4
∼ 47.4
∼ 47.4
∼ 47.4
TX 7
TX 19
TX 31
8 Results in w-iLab.t II Testbed
8.1 RSSI and ToA with Particle Filter
1
2
3
4
5
6
7
8
Measurement points in w-iLab.t I
9
10
Total
Figure 29: The results of the room accuracy in each
measurement point in the w-iLab.t I testbed
7.4 Conclusions from w-iLab.t I Experiments
An overview of the performance of the different
localization solutions is given in Table 10. As a
representative of a typical building with plywood walls,
signals are less attenuated than in the TWIST testbed,
resulting in less unique wireless features per room. As
a result, fingerprinting solutions perform worse than
in the TWIST environment. Also the ToA and RSSI
based solutions have significantly degraded performance.
This can be explained by the fact that, due to testbed
limitations, anchor nodes are not installed on the
corner points, meaning that several evaluation points
are outside the grid of anchor nodes. In addition,
although the walls are made from plywood that have
a very small attenuation factor, signal propagation still
behaves very unpredictable due to the presence of large
The results for the particle filter based localization using
RSSI and ToA range measurements are presented in
Table 11.
Table 11 Statistical information about the performance of
the particle filter algorithm in w-iLab.t II testbed
Metric
Average error [m]
Min. error [m]
Max. error [m]
Median error [m]
RMS error [m]
Response time [ms]
RSSI
ToA
6.41
0.90
20.22
5.68
8.05
59633
6.66
0.99
27.06
5.50
8.59
59620
In this testbed, the measurements are collected in a
slightly different way than in TWIST and w-iLab.t I
testbeds. A single channel is used due to a limitation
of the testbed nodes. Moreover, instead of collecting
multiple measurements from a specific testbed node
before switching to the next testbed node, a single
message is exchanged with each infrastructure node and
when all nodes have been tried, the process starts over
18
T. Van Haute et al.
14
Abs. Range Error vs. Number of Measurements
rssi
toa
Abs. Error (m)
12
10
8
6
40
100
200
300
400
# Measurements
500
600
Figure 30: The absolute range error is not affected for
neither ToA nor RSSI measurements
fewer measurements. For this reason a maximum of
approximately 900 measurements are collected from a
single node in this testbed. Figure 30 shows that, at
least for within this range of collected measurements, the
accuracy is not affected for any of the ranging methods.
Although few measurements are collected from each
node, we observe a high response time due to the fact
that many testbed nodes are used at each measurement
point. As in the w-iLab.t I testbed, the response time
can be reduced by limiting the amount of testbed nodes
used at each measurement point.
Figure 31 shows the CDFs for the range measurements
and the localization estimations. Although the ToA
range measurements (left graph) are less accurate than
that of the RSSI based method, the final localization
estimations (right graph) for the two methods are more
or less equal. The power consumptions stated for TWIST
and w-iLab.t I testbeds are also applicable here.
8.2 Fingerprinting
1.0
Abs Range Error CDF (wilab2, reference_scenario)
Table 12 contains the accuracy statistics of the
fingerprinting localization solutions described in
Section 3.2. No room accuracy is reported because
the testbed consists of a single large open space.
The accuracy is significantly lower than the accuracy
obtained in the other testbeds (Figure 32). This
degradation is mainly caused by two physical
characteristics of the environment: (i) no separate rooms
are present which makes it difficult to create unique
fingerprints for each location and (ii) due to the metal
walls, random reflections result in signal strengths that
vary strongly from packet to packet.
Probability
0.8
0.6
0.4
0.2
0.00
1.0
rssi
toa
5
15
10
Abs. Error (m)
20
25
Localization Error CDF (WiLab2, reference_scenario)
Probability
0.8
0.6
0.4
0.2
0.00
rssi
toa
5
10
Error (m)
15
20
25
Figure 31: CDFs for the absolute range error (top)
and the localization error (bottom)
again with the first node. As a result, measurements
are collected from more testbed nodes, but each having
Figure 32: CDF of the localization error of
fingerprinting based solutions in w-iLab.t II testbed
Figure 33 shows the influence of collecting additional
data before creating fingerprints. It demonstrates the
importance of using robust fingerprinting creation
methods (e.g. PH distance of RSSI quantiles) and
demonstrates that these robust fingerprinting creation
methods can be used to generate more accurate results,
Comparability of RF-based Indoor Localization Solutions in Heterogeneous Environments
on the condition that more data is collected (at the cost
of higher response delays).
19
The CDF of the localization error can be found in
Figure 34. The average accuracy is significantly lower
than in the previous environments, mainly due to the
many reflections in the environment thereby causing selfinterference. The spatial spread of the accuracy is shown
in Figure 35 using a box-plot.
35,00
Error distance [m]
30,00
25,00
20,00
15,00
10,00
5,00
0,00
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
Measurement points in w.iLab-t II
Figure 33: Fingerprint collection delay versus point
accuracy
Table 12 Statistical information about the performance
of fingerprinting algorithms in w-iLab.t II testbed
Metric
Average error [m]
Min. error [m]
Max. error [m]
Median error [m]
RMS error [m]
Response time [s]
KL
ED
PH
24.76
3.00
47.43
21.0
28.09
24.78
19.08
3.00
39.00
18.97
20.76
24.37
8.13
0.00
15.10
6.70
8.97
24.12
Figure 35: Proximity & weighted RSSI solution spatial distribution of accuracy error in w-iLab.t II,
including maximum error, minimum error, quartile 1,
quartile 2 and median error
Table 13 Accuracy of the hybrid localization algorithm in
the w-iLab.t II testbed (Tx = 31)
17.16
1.57
52.15
16.24
19.73
15.7
Average error [m]
Min. error [m]
Max. error [m]
Median error [m]
RMS error [m]
Response time [ms]
8.4 Conclusions from w-iLab.t II Experiments
8.3 Proximity & Weighted RSSI
The location accuracy of the weighted RSSI based
localization solution in w-iLab.t II testbed is shown in
Table 13.
The environment from w-iLab.t II testbeds exhibits
characteristics which are typical for many large-size
industrial indoor environments, namely open spaces
surrounded by metal obstacles, walls and ceilings.
Algorithm
Energy efficiency [mW]
Response time [ms]
Room accuracy [%]
Average Error [m]
Table 14 w-iLab.t II testbed: summarized benchmarking
results
Mobile Fixed
Particle filter solution
Spray RSSI
Spray ToA
6.41
6.66
-
59 633
59 620
∼ 105 ∼ 105
∼ 105 ∼ 105
Fingerprinting solution
Figure 34: Distribution of the RMS localization error
in w-iLab.t II testbed
KL Distance
ED Distance
PH Distance
24.76
19.08
8.13
TX Power = 31
17.16
-
∼ 24 000
∼ 24 000
∼ 24 000
∼ 7000 ∼ 500
∼ 7000 ∼ 500
∼ 7000 ∼ 500
Hybrid solution
-
15.7
∼ 57.6 ∼ 47.4
20
T. Van Haute et al.
The results clearly indicate that all tested types of RFbased localization solutions degrade significantly in these
environments. All the signals have a lot of reflections
with the metal construction, causing a lot of multipath
effects. This indicates that accurate indoor localization
in industrial open environments is a difficult task. In
this industrial environment, the ToA and RSSI based
ranging solutions using the particle filter contains the
best results. An average error distance of 6 - 7 m instead
of 11 m for fingerprinting and 17 m for the hybrid
technique.
outliers) or setting the minimum number of beacons
that is collected for location estimation can significantly
influence the performance.
As such, to allow objective comparison of localization
solutions, it is clear that independent evaluation
procedures should be defined by an impartial third party
and that such evaluation procedures should include at
least the following aspects: (i) definition of a wide set of
evaluation metrics, (ii) clear definition of the evaluation
environments in which the results are valid (iii) an
objective method for generating a representative set of
evaluation points.
9 Discussion
Over the course of the performed experiments, several
lessons were learned. First of all, the experiments
clearly show the importance of choosing representative
measurement locations. Several locations have consistent
lower accuracy results. For example the hallways (which
are narrow and as such have very low room accuracy).
As such, it is clear that the localization points should
include a representative mix of “easy to locate positions”
and more challenging ones. Ideally, a fine-grained gridlike approach should be used in which the positioning
accuracy is evaluated every X meter. All experiments in
this paper use the same evaluation points.
Another lesson learned is that the location accuracy
differs strongly between different testbeds. Environment
specifics (such as metal ceilings) strongly influence
propagation behavior. The highest accuracy was
obtained in more “traditional” brick-wall office scenarios,
such as represented by the TWIST testbed. The wiLab.t I testbed, which has plywood walls and metal
ceilings, has a lower accuracy. Finally, w-iLab.t II testbed
consists of a fully shielded environment, in which the
walls and ceilings are from metal, and contains a
number of metal obstructions. Performing localization
in this testbed, i.e. in a confined and strongly reflecting
environment, proves to be very challenging.
Thirdly, the experiments and benchmarking results
that are executed illustrate the need for evaluating a
broad set of the metrics. Although the accuracy of the
fingerprinting solutions in office environments is shown
to be very good, these solutions require significantly
more time to collect beacons for fingerprinting, which
strongly influences the response delay. Similarly, the
WiFi based fingerprinting localization solutions perform
very well, but have higher energy requirements than the
solutions using sensor nodes.
Moreover, the experiments indicate that the performance
of localization solutions strongly depends on several
algorithmic and deployment aspects, such as the
used technology, the ranging approach, the location
estimation approach, post- and preprocessing, anchor
positions, etc. Making even minor changes to one of
these aspects can have a profound influence on several
performance metrics. It was also shown that the internal
configuration of the algorithms, such as preprocessing
the data (such as removing the 10 % highest and lowest
10 Conclusion
Although many indoor localization solutions exist, this
paper pointed out that the scientific evaluation methods
and praxis for RF based solutions are currently limited
in scope. Localization solutions are evaluated mainly
based on location accuracy and are evaluated in a single
testbed environment. As a result, it is not clear to which
level the results from existing scientific literature can be
compared to each other.
To evaluate how these different conditions can influence
the localization performance, three localization solutions
were selected that represent typical approaches for
indoor localization, including multiple technologies
(IEEE 802.11 and IEEE 802.15.4), multiple localization
approaches (fingerprinting, time-of-arrival and RSSIbased) and multiple processing methods. To allow
objective comparisons, the same evaluation methodology
was used to evaluate the performance of these
localization solutions in three different environments:
an office environment with brick walls, an office
environment with plywood walls and an open
environment with metal walls and metal obstacles.
The main conclusion of these experiments were the
following.
• Several inherent trade-offs between different metrics
have been identified, which are typically ignored when
reporting only on the accuracy of the solutions. More
specifically, the results show a very clear trade-off
between the collected number of measurements (which
are directly translated into energy consumption and
response delay) and the point-level accuracy.
• The accuracy of localization solutions depends
strongly on the characteristics of the environment.
Due to the presence of concrete and/or brick walls
in the office testbeds, different rooms are very
diverse in terms of their wireless characteristics.
Localization solutions that exploit this diversity,
such as fingerprinting based approaches, obtain the
highest accuracy. In contrast, in more industriallike open environments time-of-arrival solutions
performed better. These results show that future
scientific literature describing performance results
of localization solutions should include detailed
descriptions of the used evaluation environment(s),
Comparability of RF-based Indoor Localization Solutions in Heterogeneous Environments
including
information
such
as
propagation
characteristics, typical room sizes and a description
of the materials used in walls and ceilings.
• We have shown that the choice of evaluation points
strongly influence the reported accuracy. As such,
papers that use self-selected evaluation points can
significantly influence their reported accuracy by
artificially selecting those evaluation points that
outperform other locations.
• Due to testbed constraints, one evaluation
environment contained evaluation points outside the
grid of anchor points. It was shown that this set-up
had a negative influence on some of the evaluated
solutions (mainly the RSSI-based and ToA solutions)
but not on the fingerprinting solution. As such, when
considering which is the best localization solution for
an industrial deployment, building layout constraints
should also be included.
• The accuracy can decrease significantly when
evaluating an environment for which the localization
solution is not specifically tweaked. For example, all
tested solutions suffered from degraded accuracy in
the open industrial-like environment, up to a factor
10 lower. Since most existing solutions have been
optimized for office environments, these results hint
that many existing localization solutions might not
be ready for use in industrial environments or other
challenging environments, such as underground mines.
The above findings reveal several weaknesses in
the evaluation methods used in the majority of
existing scientific literature of indoor localization
solutions. As such, there is a clear need for a
standardized evaluation methodology to objectively
compare different localization solutions in multiple
conditions, as developed within the EVARILOS project
and pursued by standardization agencies like ISO.
Acknowledgments
The research leading to these results has received
funding from the European Union’s Seventh Framework
Program (FP7/2007-2013) under grant agreement no
317989 (STREP EVARILOS). The author Filip Lemic
was partially supported by DAAD (German Academic
Exchange Service).
References
[1] Bulusu, N.; Heidemann, J.; Estrin, D., GPS-less Low
Cost Outdoor Localization For Very Small Devices
21
[4] Liu, H., Darabi, H., Banerjee, P. (2007) ‘Survey of
wireless indoor positioning techniques and systems’,
IEEE Transactions on systems, Vol. 37, No. 6
[5] ‘The EVARILOS project’, http://www.evarilos.eu
[6] Van Haute, T., De Poorter, E., Lemic, F., Wirström, N.,
et al (2013) The EVARILOS Benchmarking Handbook:
Evaluation of RF-based Indoor Localization Solutions,
MERMAT workshop
[7] Van Haute, T., De Poorter, E., Lemic, F., Wirström, N.,
et al (2013) D2.1: EVARILOS Benchmarking Handbook,
The EVARILOS project
[8] Van Haute, T., De Poorter, E., Lemic, F., Wirström,
N., et al (2013) D2.2: Report on experiments without
interference, The EVARILOS project
[9] Lemic, F., Wirström, N., Handziski, V., et al
(2013) D2.3: Report on experiments with controlled
interference, The EVARILOS project
[10] ISO Standard - ISO/IEC CD 18305 - Information
technology – Real time locating systems – Test and
evaluation of localization and tracking systems http://www.iso.org/iso/home/store/catalogue_
htc/catalogue_detail.htm?csnumber=62090
[11] EvAAL - Evaluating AAL Systems through Competitive
Benchmarking - http://evaal.aaloa.org/index.
php?option=com_content&view=article&id=187:
technical-annexes-localization2013&catid=15&
Itemid=261
[12] Dezhong Y., Chen Y., Anind K. D., Christian K.,
Geyong M., Laurence T. Y., Hai J. (2014) ‘Energy
efficient indoor tracking on smartphones’, Future
Generation Computer Systems 39, pp 44–54
[13] Veljo Otsason, Alex Varshavsky, Anthony LaMarca
and Eyal de Lara, Accurate GSM Indoor Localization,
UbiComp 2005: Ubiquitous Computing pp 141–158
[14] Andrew M. Ladd, Kostas E. Bekris, Algis P. Rudys, Dan
S. Wallach, and Lydia E. Kavraki, On the Feasibility of
Using Wireless Ethernet for Indoor Localization, IEEE
Transactions on Robotics and Automation, VOL. 20,
NO. 3, JUNE 2004, pp. 555–559
[15] Krishna Chintalapudi, Anand Padmanabha Iyer
and Venkata N. Padmanabhan, ‘Indoor Localization

Without the Pain’, Microsoft
research
[16] Eladio Martin, Oriol Vinyals, Gerald Friedland and
Ruzena Bajcsy (2010), Precise Indoor Localization Using
Smart Phone, MM ’10 Proceedings of the international
conference on Multimedia, pp. 787–790
[2] Hightower, J. and Borriello, G. (2001) ‘Location systems
for ubiquitous computing’, Computer, Vol. 34, No. 8
[17] Sunkyu W., Seongsu J., Esmond M., Linyuan X.,
Changsu C., Muwook P., Joon H. (2011) ‘Application of
WiFi-based indoor positioning system for labor tracking
at construction sites: A case study in Guangzhou MTR’,
Automation in Construction, pp 3–13
[3] Pahlavan, K., Li, X. and Makela, J. (2002) ‘Indoor
geolocation science and technology’, IEEE Commun.
Mag., Vol. 40, No. 2, pp.112–118
[18] Stella M., Russo M., Begušić D. (2014) ‘Fingerprinting
based localization in heterogeneous wireless networks’,
Expert Systems with Applications, pp 6738–6747
22
T. Van Haute et al.
[19] Steiner C. and Wittneben A. (2011) ‘Efficient Training
Phase for Ultrawideband-Based Location Fingerprinting
Systems’, IEEE Transactions on Signal Processing, vol.
59, no. 12, pp 6021–6032
[20] Elson, J. and Römer, K. (2003) ‘Wireless sensor
networks: A new regime for time synchronization’, ACM
SIGCOMM CCR, Vol. 33, No. 1, pp.149–154
[21] Mazomenos, E.,De Jager, D., Reeve, J. and White,
N. (2011) ‘A two-way time of flight ranging scheme
for wireless sensor networks’, European Conference on
Wireless Sensor Networks (EWSN)
[22] Pettinato, P., Wirström, N., Eriksson, J. and Voigt
T. (2012) ‘Multi-channel two-way time of flight sensor
network ranging’, Proceedings of the 9th European
conference on Wireless Sensor Networks (EWSN’12),
978-3-642-28168-6, pp.163–178
[23] Wirström, N., Misra, P. and Voigt, T (2014) ‘Spray:
A multi-modal localization system for stationary sensor
network deployment’, Wireless On-demand Network
Systems and Services (WONS), 2014 11th Annual
Conference on, pp. 25–32
[24] Lemic, F. (2014) Benchmarking of Quantile-based
Indoor Fingerprinting Algorithm, TKN Technical Report
TKN-14-001
[25] Lemic, F., Behboodi, A., Handziski, V., Wolisz, A.
(2014) ‘Experimental Decomposition of the Performance
of Fingerprinting-based Localization Algorithms’,
International Conference on Indoor Positioning and
Indoor Navigation, IPIN’14
[26] Milioris, Dimitris, et al (2011) ‘Low-dimensional SignalStrength Fingerprint based Positioning in Wireless
LANs’, Ad Hoc Networks
[27] Chambers, John M., et al (2006) ‘Monitoring Networked
Applications with Incremental Quantile Estimation’,
Statistical Science 21.4, pp.463–475
[28] Van Haute, T., Rossey, J., Becue, P., De Poorter,
E., Moerman, I., Demeester, P. (2014) ‘A hybrid
indoor localization solution using a generic architectural
framework for sparse distributed wireless sensor
networks’, Federated Conference on Computer Science
and Information Systems, WSN2014, pp.1015–1021
[29] Handziski, V., Köpke, A., Willig, A., Wolisz, A. (2006)
‘TWIST: A Scalable and Reconfigurable Testbed for
Wireless Indoor Experiments with Sensor Network’,
International Workshop on Multi-hop Ad Hoc Networks:
from Theory to Reality, RealMAN06
[30] Lemic, F., Büsch, J., Chwalisz, M., Handziski,
V., Wolisz, A. (2014) ‘Demo Abstract: Testbed
Infrastructure for Benchmarking RF-based Indoor
Localization Solutions under Controlled Interference’,
European Conference on Wireless Sensor Networks,
EWSN’14
[31] Lemic, F., Büsch, J., Chwalisz, M., Handziski, V.,
Wolisz, A. (2014) ‘Infrastructure for Benchmarking
RF-based Indoor Localization under Controlled
Interference’, International Conference on Ubiquitous
Positioning, Indoor Navigation and Location-Based
Services, UPINLBS’14
Tom Van Haute is a doctoral researcher at Ghent
University. He received his M. Sc. degree (cum laude)
in Computer Science Engineering from Ghent University,
Belgium, in 2012. In September 2012, he joined the
Department of Information Technology (INTEC) at Ghent
University. Within this department, he is working at the
Internet Based Communication Networks and Services
research group (IBCN). His research is focused on wireless
sensor networks combined with indoor localization and
indoor navigation in particular. Additionally, he is providing
guidance to Computer Science Engineering Master theses
and “Informatics” course.
Dr. Eli De Poorter is a postdoctoral researcher at Ghent
University. He received his master degree in Computer
Science Engineering from Ghent University, Belgium, in
2006. He received his Ph.D. degree in 2011 at the Department
of Information Technology at Ghent University through
a Ph.D scholarship from the Institute for Promotion of
Innovation through Science and Technology in Flanders
(IWT-Vlaanderen). After obtaining his PhD, he received a
FWO postdoctoral research grant and is now a post-doctoral
fellow at the same research group, where he is currently
involved in and/or research coordinator of several national
and international projects. His main research interests
include wireless network protocols, network architectures,
wireless sensor and ad hoc networks, future internet, selflearning networks and next-generation network architecture.
He is part of the program committee of several conferences
and is the author or co-author of more than 50 papers
published in international journals or in the proceedings
of international conferences. He is also the creator of the
patented IDRA architecture (http://idraproject.net),
a flexible communication framework for heterogeneous
networked devices.
Ingrid Moerman received her degree in Electrical
Engineering (1987) and the Ph.D degree (1992) from
the Ghent University, where she became a part-time
professor in 2000. She is a staff member of the research
group on Internet-Based Communication Networks and
Services, IBCN (www.ibcn.intec.ugent.be), where she is
leading the research on mobile and wireless communication
networks. Since 2006 she joined iMinds, where she is
coordinating several interdisciplinary research projects.
Her main research interests include: Sensor Networks,
Cooperative and Cognitive Networks, Wireless Access, SelfOrganizing Distributed Networks (Internet of Things) and
Experimentally-supported research. Ingrid Moerman has a
longstanding experience in running national and EU research
funded projects. At the European level, Ingrid Moerman is
in particular very active in the FP7 FIRE (Future Internet
Research and Experimentation) research area, where she is
coordinating the CREW project and further participating
in IP Fed4FIRE, STREP EVARILOS, STREP FORGE and
IP FLEX. In the FP7 research area on Future Networks,
she is involved in IP LEXNET and STREP SEMAFOUR.
Ingrid Moerman is author or co-author of more than
600 publications in international journals or conference
proceedings. She is associate editor of the EURASIP Journal
on Wireless Communications and Networking and vice-
Comparability of RF-based Indoor Localization Solutions in Heterogeneous Environments
president of the expert panel on Informatics and Knowledge
Technology of the Research Foundation Flanders (FWO).
Filip Lemic is a junior researcher and a PhD candidate at
the Telecommunication Networks Group at the Technical
University of Berlin. He finished his bachelor and master
studies at the Faculty of Electrical Engineering and
Computing at the University of Zagreb. His main scientific
interests are in context awareness, with an emphasis on the
indoor localization.
Dr.-Ing. Vlado Handziski is a senior researcher in the
Telecommunication Networks Group at the Technische
Universität Berlin, where he coordinates the activities
in the areas of sensor networks, cyber-physical systems
and internet of things. He is currently also serving as
interim professor at the chair for Embedded Systems at
Technische Universität Dresden. He received his doctoral
degree in Electrical Engineering from TU Berlin (summa
cum laude, 2011) and his M.Sc. degree from Ss. Cyril and
Methodius University in Skopje (2002). Dr. Handziski’s
research interests are mainly focused on testing and software
architecture aspects of networked embedded systems. He
has participated and led testbed infrastructure activities
in several large European research projects like Embedded
WiSeNts, CONET, EVARILOS and EIT ICT Labs. He is
core developer of TinyOS, a popular operating systems for
sensor networks, and chief architect of the TWIST testbed.
At TU Berlin, he teaches the master-level courses on“Ad Hoc
and Sensor Networks” and “Networked Embedded Systems”
and he teaches the undergraduate course “Informatics 1” at
TU Dresden. Dr. Handziski is member of IEEE and ACM.
Adam Wolisz received his degrees (Diploma 1972, Ph.D.
1976, Habil. 1983) from Silesian University of Technology,
Gliwice, Poland. He joined TU-Berlin in 1993, where he
is a chaired professor in telecommunication networks and
executive director of the Institute for Telecommunication
Systems. He is also an adjunct professor at the Department
of Electrical Engineering and Computer Science, University
of California, Berkeley. His research interests are in
architectures and protocols of communication networks.
Recently he has been focusing mainly on wireless/mobile
networking and sensor networks.
Niklas Wirström is a researcher in the Networked
Embedded Systems (NES) group at SICS and a PhD student
at Uppsala Universitet, Sweden. His research focus is on
Machine Learning techniques for localization in WSN and
other resource constrained systems.
Thiemo Voigt is professor at Uppsala University. He also
leads the Networked Embedded Systems group at SICS
Swedish ICT. His main interests are networking and system
issues in wireless sensor networks and the Internet of Things.
He has published papers at flagship sensor networking
conferences such as ACM SenSys and IEEE/ACM IPSN and
received awards for several of these publications. He has also
been TPC co-chair for IEEE/ACM IPSN and EWSN, the
European Conference on Wireless Sensor Networks as well
as track chair for IEEE DCOSS, IEEE MASS and IEEE
ICDCS. He is on the editorial board for the ACM Transaction
on Sensor Networks and the IEEE IoT Newsletter.
23