Support Us
FactWire has discovered a facial detection feature in the source code of the government’s LeaveHomeSafe mobile app, although it was unable to confirm whether the feature has been activated or if it is currently in use.
In response to FactWire’s inquiry, the government admitted to the existence of a facial detection feature in the application’s source code but claimed that it has never been used. It said that it had already asked the application’s developers – a contractor firm – to study the feasibility of removing the facial detection source code without affecting the “normal operation” of the application.
FactWire examined the source code of the latest Android release of the LeaveHomeSafe app (3.2.0), along with six other previous versions by converting the ‘classes.dex’ files – obtained from the apk files downloaded from the app’s official website – into human-readable java source files.
The converted source file comprises some 20 folders. One of those, named “org”, contains a subfolder named “reactnative” which further comprises three folders titled “facedetector”, “camera”, and “maskedview” respectively.
React Native is an open-source software framework for developers to incorporate into their applications. These folder names and the code found in the “camera” folder suggest that the LeaveHomeSafe app adopted code from a React Native codebase called “react-native-camera”, which drives mobile device cameras.
However, most of the modules, functions and classes in the java code have been renamed, leaving only alphanumeric code (such as “f2” and “i6”) in their place using a technology called “code obfuscation”. The technology is commonly used to protect applications’ copyright and to avoid them being cracked. Three of the four java files under the “facedetector” folder of the LeaveHomeSafe app have been so ‘obfuscated’.
To understand the use of this obfuscated code in the LeaveHomeSafe app, FactWire compared it against the open-source React Native code and found that the structure of the obfuscated code matched that of the codebase known as “react-native-camera-master”.
One of the java files, originally known as “FaceDetectorUtils.java” but renamed “a.java” in LeaveHomeSafe, may be used to detect the positions of a person’s mouth, nose tip, left and right cheeks, eyes, ears and earlobes. It is also able to detect a subject’s head tilt in degrees and calculate the probability that they are smiling or has each eye open.
The facial detection module can be found in all versions of the LeaveHomeSafe app that FactWire examined, including Android versions 1.0.4, 1.0.5, 2.1.4, 2.1.5, 3.0.2, 3.1.0 and the latest, 3.2.0, released on April 18 of this year, suggesting that the module has existed since the app’s early launch in 2020. This is, however, not mentioned on the app’s official website, nor in mobile app store descriptions.
Technical specifications published on the LeaveHomeSafe website state that the React Native framework was used to develop the app. Since the same framework and codebase can be used to write Android and iOS apps, the codebase of the Android and iOS versions of LeaveHomeSafe are thought to be largely similar.
FactWire has examined the code of the Android releases, but not the application built for iOS. Version 3.2.1 was released on April 30 for iOS, but the latest version for Android remains 3.2.0 as at the time of writing.
Marc Rousavy, who personally maintains the “react-native-camera” module and whose app development and tech consultancy firm Margelo was responsible for writing React Native’s core, confirmed to FactWire after evaluating LeaveHomeSafe’s code that a facial detection module can be found in the app, since it employs the “react-native-camera” module. However, it is difficult to tell whether the app’s facial detection module is actually active because the code in “index.android.bundle” (which contains information on how the app runs) has been encoded into non-human-readable Hermes bytecodes.
Although Hermes bytecodes are hardly readable, Rousavy found that the identifier of a function named “onFaceDetected” is passed to another function within the bundle of the 3.1.0 version of the app, which might mean that something is written to the device’s disk when a face is detected, although what that is, exactly, remains unclear.
FactWire has found that that same part of the 3.2.0 bundle has been slightly modified. The “onFaceDetected” function may be reading some kind of information instead when the function itself is called or when its identifier is passed to another function.
Leo To, a software engineer who has worked for the government as a contractor, believes that the reason a facial detection module can be found in the LeaveHomeSafe app is that it simply originally existed in the ready-made “react-native-camera” module, and had not been removed by the app developer when adopting the module. Apart from detecting human faces, the same module can be used to detect barcodes and QR codes as well, he explained.
In To’s opinion, however, unused modules should be removed from the application’s source code, as it might create operational problems. “It is definitely possible to import the entire module and remove just the facial detection part,” he said, “The developers from Cherrypicks (the contractor) have, of course, the technical ability to do it. It is just a question of whether they decide to do it.”
He also suspects that the Office of the Government Chief Information Officer (OGCIO) may not even know about the code, relating that in his own experience, the government does not inspect the source code, despite requiring contractors to submit it. “They would only ask you to fix certain problems that they encounter when testing the app,” To said.
There is no need for the app to ask for extra permissions on the device to activate the facial detection feature as the app can record a video stream by only using the camera (which it already has permission to use to read QR codes and car licence plates). There is no indication as to how the video would be used in the system’s backend, To explained.
FactWire tested the LeaveHomeSafe app by entering debug mode using Android Studio and a smartphone and learned that the app currently uses only “camera ID 0” – that is, the rear camera of the device – but not the front-facing camera, which would be much more likely to capture the user’s face. The probability of the user’s face being detected is thus very low.
In a reply to FactWire’s inquiry, the OGCIO confirmed that Cherrypicks, the firm contracted to develop the LeaveHomeSafe app, used the “react-native-camera” module to allow the app to scan taxi car licence plates and the QR codes displayed at certain venues and on vaccination records. It admitted that the module includes a facial detection feature, but said that it had not been aware of it before receiving FactWire’s inquiry and that Cherrypicks had never activated it, adding that such a feature is unnecessary for the operation of the app.
The government said it had immediately asked Cherrypicks to “study the feasibility of removing the facial detection module while ensuring the app’s normal operation would not be affected,” in order to eliminate unnecessary concerns from the public.
According to the government’s contract with Cherrypicks Limited, the LeaveHomeSafe app and its source code are the intellectual property of the government. However, the government does not own the source code that the app draws from, such as that already used in Cherrypicks’ previous products and ready-made commercial or open-source codebases and modules.
Addressing privacy concerns, the OGCIO said that there was a need for the app to obtain certain information from the mobile device including the app’s version, its operating system, the device model, Android API level, and to check whether there is a passcode or biometric verification set up in the case of importing a vaccination record.
“We treat the public’s privacy concerns seriously by consulting the Office of the Privacy Commissioner for Personal Data every time the LeaveHomeSafe app is updated with new features, to ensure that it complies with the Personal Data (Privacy) Ordinance,” reads the reply.
The government emphasised that the app is “safe and reliable” by stressing that its privacy impact and security risk have been assessed and audited by an independent third party. However, both the security risk and privacy impact reports, which can be found online, do not mention inspecting the idle modules of the app, despite the privacy impact assessment stating that its aim was “to identify and address any data privacy implications/issues.”
Cherrypicks Limited did not respond to FactWire’s inquiry, only referring the questions to the OGCIO.
An anti-epidemic measure, the LeaveHomeSafe application has been available for download since November 16, 2020, and supports the iOS, Android and Huawei operating systems. Members of the public can use the application to record when they enter and leave certain venues by scanning the venues’ QR codes.
The application’s developer, Cherrypicks Limited, is a subsidiary of the Chinese online gaming company NetDragon Websoft Holdings Limited (HK.777).
According to a written OGCIO reply to a Legislative Council inquiry on April 13, the estimated cost thus far of the LeaveHomeSafe app is about HKD 8.6 million, of which about HKD 3.6 million went to the maintenance and upgrade of the application and its backend systems, and the remaining HKD 5 million to its support and operation, including staff salaries.
The Secretary for Innovation and Technology Alfred Sit announced on April 13 that a new version, 3.2.0, was expected to be released in early May, with a new feature to import recovery records. This, the current version, was released on April 18, five days following the announcement.
