Research.

Research is a part of AiFi's DNA. We are constantly innovating and disrupting the status quo. AiFi is collaborating with Carnegie Mellon University to lead the way in defining what is Autonomous Retail.

Competitions.

March 22
CPS-IoT Week has announced that due to COVID-19 it will go fully virtual on 21-24 April. As a consequence, the AutoCheckout Competition will also go fully virtual and will maintain its dates. 21-22 April, 2020.
For more information please read the UPDATED Logistics Details

For further information there will be another Clarification Conference Call
scheduled for Thursday, March 26 2020 @ 8:30am PDT (Time reserved for: 1h).
To join the video meeting, click this link: https://meet.google.com/grw-zgzn-dtb Otherwise, to join by phone, dial +1 478-419-0123 and enter this PIN: 277 667 475

March 10
Thank you all the participants that joined the Clarification Conference Call, here are the updates based on the questions that came up during the call:
- Camera placement file: here
- More detail specifications are at the bottom of the README - Here
- Please delay the travel plans until we have a confirmation of the competition

March 6
Clarification Conference Call
scheduled for Tuesday, March 10th 2020 @ 8:45am PST (Time reserved for: 1h15m). If you have questions join the video meeting by clicking this link: https://meet.google.com/grv-whwk-esw
Otherwise, to join by phone, dial +1 848-667-8684 and enter this PIN: 914 936 458#

March 4
The participating teams have been announced! We have 7 teams including 1 industry participant. Please check out the Teams section.

Feb 19

Submit your abstract with a proposed approach even if you have not had the chance to run your approach agains the testing samples. The organizing team will help evaluate the approach and provide feedback on how to run the approach.Thank you all the participants that joined the Clarification Conference Call, here are the updates based on the questions that came up during the call:
- Camera placement file: here
- More detail specifications are at the bottom of the README - Here
- Please delay the travel plans until we have a confirmation of the competition

March 6
Clarification Conference Call
scheduled for Tuesday, March 10th 2020 @ 8:45am PST (Time reserved for: 1h15m). If you have questions join the video meeting by clicking this link: https://meet.google.com/grv-whwk-esw
Otherwise, to join by phone, dial +1 848-667-8684 and enter this PIN: 914 936 458#

March 4
The participating teams have been announced! We have 7 teams including 1 industry participant. Please check out the Teams section.

Feb 19

Submit your abstract with a proposed approach even if you have not had the chance to run your approach agains the testing samples. The organizing team will help evaluate the approach and provide feedback on how to run the approach.

Autonomous retail has the potential to change the way people perceive shopping in a similar way e-commerce did. Autonomous stores could offer the convenience of 24/7 operation close to the customer, eliminate friction (e.g. waiting in line to pay), monitor stock in real-time and better understand human shopping behavior. In recent years, several automated retail technologies have been proposed. However accuracy and cost effectiveness of these approaches have been a major bottleneck preventing large scale deployments and their study. This competition aims to bring industry and academia closer together by reducing the barrier of entry for researchers to access data and infrastructure. This will allow the community to design new approaches and compare their performance under similar conditions.

What will you receive:

- A video feed from 12 cameras inside the store.
- 3D positon of all humans inside the store.
- Weight Sensors data from all sensors on the shelves.
- A trigger that someone entered/exited the store.
- Layout of the sensors and cameras.
- Layout of the products in the store.
- Detailed information of the products.

What will you compute:

Upon receiving the trigger of a person exiting the store you will provide a list of products that the person has exited with.


See details HERE

Eligibility
Both academia and industry submissions are encouraged. All techniques, such as vision-only, sensors only, or sensor fusion, are welcome, except those that require humans’ manual interaction. Contesters will be able to test their algorithms using the "Testing Data". During the competition contesters will have the opportunity to deploy their system and test it a day before the evaluation day (This might be adjusted depending on the number of participants). The results will be shown and processed in the stores servers and infrastructure.

Demo submissions that do not meet one or more of the guidelines above will be included in the poster session and will be evaluated as a regular submission, but they will not be considered for prizes.

The competition will take place if at least 5 teams respond to this preliminary call for competition.
Technology Used
The store will be using color video cameras and depth video cameras. The store has 4 gondolas with 5 shelves each. Each shelf has 12 weight sensing plates.

All sensing modalities are synchronized using NTP which will guarantee time synchronization within tens of milliseconds.

More details of the testbed will be under the "Sample Data" Section.
Evaluation and Prizes
The results are based on an F1 score of the receipts generated by each team. A receipt is considered correct only if all items in the estimated receipt match the all items in the ground truth receipt.
An award will be given to the top 3 teams. When F1 ties, latency of the response will be used for tie breaking. The winning teams will receive a cash award.
Poster Session
A poster session dedicated to all competition participants will be organized during the conference. Participants will have the opportunity to explain their system to conference attendees.
Submission Guidelines
Contesters must submit an abstract describing their approach and deployment requirements by the contest registration deadline. Submissions are treated as confidential until the competition. Submissions must be at most two (2) single-spaced 8.5″ x 11″ pages, including figures, tables, and references. Submission should follow the exact same format as regular, full IPSN 2020 papers. Abstracts should include the names and affiliations of all authors.

Templates can be found here.

Abstracts should be sent over email to: joao@aifi.io on or before February 28th 2020 with the following subject line: 2020 AiFi Nanostore AutoCheckout Competition Submission.

UPDATED DUE TO COVID-19
More details about the competition can be found HERE

  • April 21

    08:30 - 17:00
    Setup Day. All teams meet virtually through google meets at 8:30am to go over the rules and setup.
  • April 22

    08:30 - 17:00
    Evaluation Day. Shopping time! All teams are evaluated during the entire day.
  • April 23

    11:30 - 12:30
    Official Results Announcement

Team 1:
- Authors:
Zhang et al.
- Title: An improved multimodal fusion technique for Cashier-Less Stores
- Affiliation: Harbin Institute of Technology, China

Team 2:
- Authors:
Mohammadi et al.
- Title: Advanced Video Processing for Efficient data Analytic
- Affiliation: University of Georgia, U.S.

Team 3:
- Authors:
Bao et al.
- Title: Multi-Person Shopping (MPS) for Cashier-Less Store
- Affiliation: Carnegie Mellon University, U.S.

Team 4:
- Authors:
Ortiz et al.
- Title: Accurately Aggregating Relevant Target User and Time Based Data
- Affiliation: H-E-B, U.S.

Team 5:
- Authors:
Zhang et al.
- Title: Location-aware multi-modal sensor fusion for computational efficient autonomous inventory monitoring system
- Affiliation: UC Merced, U.S.

Team 6:
- Authors:
Gao et al.
- Title: Autonomous Checkout for Retail Store--Multi-customer Monitoring
- Affiliation: Stanford University, U.S.

Team 7:
- Authors:
Ashok et al.
- Title: Uni-Modal Sensing using Embedded WeightSensors for Fully-Autonomous Store Checkout
- Affiliation: Georgia State University, U.S.

In order to get you started our organizers have gone shopping.
Please follow this repository to get started on how to use the dataset.

Camera Placement file:
Intrinsics and extrinsics - Here

Simple Example:
Video Data - Here (17.1mb)
Dataset (without depth images) - Here (239 mb)
Complete Dataset (with depth images) - Here (2.0 gb)

More data:
Test 2   - Download
Test 3   - Download
Test 4   - Download
Test 5   - Download
Test 6   - Download
Test 7   - Download
Test 8   - Download
Test 9   - Download
Test 10 - Download
Test 11 - Download
Test 12 - Download
Test 13 - Download
Test 14 - Download
Test 15 - Download
Test 16 - Download
Test 17 - Download
Test 18 - Download
Test 19 - Download
Test 20 - Download
Test 21 - Download
Test 22 - Download
Test 23 - Download
Test 24 - Download

  • Date
    April 21, 2020 - April 22, 2020
  • Location
    Remotely through Google Meets
  • Registration Deadline
    Submit 2 page abstract by Feb 28, 2020
  • Contact
    João Diogo Falcão: joao@aifi.io
  • Sponsors
  • Organizers
    João Diogo Falcão
    (AiFi Research & Carnegie Mellon University)
    Carlos Ruiz
    (AiFi Research)
    Hae Young Noh
    (Stanford University)
    Pei Zhang
    (Carnegie Mellon University)
    Shijia Pan
    (UC Merced)

Conferences.

Papers.

Autonomous Inventory Monitoring through Multi-Modal Sensing (AIM3S) for Cashier-Less Stores

@inproceedings{ruiz2019aim3sDemo,
 title={Demo Abstract: Autonomous Inventory Monitoring through Multi-Modal Sensing (AIM3S) for Cashier-Less Stores},
 author={Ruiz, Carlos and Falcao, Joao and Pan, Shijia and Noh, Hae Young and Zhang, Pei},
 booktitle={Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation},
 pages={395--396},
 year={2019},
 organization={ACM}
}

Autonomous Inventory Monitoring through Multi-Modal Sensing for Cashier-Less Convenience Stores

@inproceedings{ruiz2019aim3s,
 title={AIM3S: Autonomous Inventory Monitoring through Multi-Modal Sensing for Cashier-Less Convenience Stores},
 author={Ruiz, Carlos and Falcao, Joao and Pan, Shijia and Noh, Hae Young and Zhang, Pei},
 booktitle={Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation},
 pages={135--144},
 year={2019},
 organization={ACM}
}

Datasets.

Coming Soon. Subscribe below to be notified.

Thank you! We will notify you once the dataset is public.
Oops! Something went wrong while submitting the form.
Ⓒ 2019 AiFi Inc.