provide an option to handle unintroduced APIs as weak symbols

[DanAlbert]

Tracking bug for the roadmap item "Weak symbols for API additions": https://android.googlesource.com/platform/ndk/+/master/docs/Roadmap.md#weak-symbols-for-api-additions

iOS developers are used to using weak symbols to refer to function that may be present in their equivalent of targetSdkVersion but not in their minSdkVersion. They use a run-time null check to decide whether the new function is available or not. Apparently clang also has some support for emitting a warning if you dereference one of these symbols without a corresponding null check.

This seems like a more convenient option than is currently available on Android, especially since no currently shipping version of Android includes a function to check which version of Android you're running on.

We might not want to make this the default (because it's such a break with historical practice, and might be surprising), but we should offer this as an option.

An interesting technical problem here will be dealing with the DT_NEEDED situation for “I need this library (but it might not exist yet)”

I spent a bit of last week investigating possible solutions to that last part and haven't found anything that will work on existing Android versions, so it may be that APIs added in new libraries either can't be covered by this or will need to be handled in a different manner (a wrapper library or something like that).

[alexcohn]

Consider building a set of fallback shim libraries. If my system does not resolve some API, the dummy entry in shim library will be dynamically linked instead. You will need a set of static libs for each pair of (compileSdkVersion, minSdkVersion). Actually, it's enough to have one lib per (compileSdkVersion, minSupportedSdkVersion) for given NDK release.

Unlike weak symbols, this workaround does not require runtime check of nullptr, but still the app is expected to misbehave if it actually performs a call to a function that is not actually implemented. From the programmer's point of view, this works similar to the Java SDK.

As for DT_NEEDED, it can be resolved with similar dummy shim libraries. The lookup order on recent Androids favors system folder over the app folder, so if the APK supplies, e.g. libneuralnetworks.so.so, it will be loaded only on Nougat and below.

[DanAlbert]

Yeah, something along those lines is more or less what we landed on after discussing this yesterday. One of the original motivations for trying to do this via weak symbols is that they have a few nice properties.

The first is that Clang's __attribute__((introduced=21)) can be configured to automatically create a weak symbol when the target API level predates the introduced API level.

The second is that using some maybe-not-available API would look like the following:

if (some_api != nullptr) {
  some_api();
} else {
  fallback_impl();
}

And supposedly Clang is able to generate a warning if a maybe-not-available API is used without a guard, which covers my concern that this is just going to give people a false sense of security when turning their build failures into run time failures on devices they don't test. Aside from my concerns that the warning would not be reliable enough (or alternatively too noisy and just disabled by developers), enh pointed out that it works nicely for single calls like the above but poorly for what is probably a more common use case:

if (new_api_1 && new_api_2 && new_api_3) {
  new_api_1();
  new_api_2();
  new_api_3();
} else {
  fallback();
}

For the latter it really would be preferable to just write the single if (device_api() >= required_api) and be done with it.

Plus, with weak symbols any error in checking for the existence of an API would just result in a segfault when the user tries to call a null function. With a shim library we can at least log something useful before aborting.

Given that (and the fact that we have yet to find a way to make the weak symbol option work at all with the case of unavailable libraries), I think a stub library is the way to go.

[wang-bin]

The second is that using some maybe-not-available API would look like the following:

if (some_api != nullptr) {
some_api();
} else {
fallback_impl();
}

For apple platforms it's not recommended, instead we can use

if (__builtin_available(iOS 11.0, macOS 10.13, *)) {
   new_api();
} else {
  ...
}

Letting the macro support android should be a good choice.

[DanAlbert]

Another issue I brought up on #1262 that isn't mentioned here is that there's no non-infectious way to set ANDROID_API_WEAK_AVAILABILITY or whatever we'd call it. Say you have your code (foo.cpp) and a dependency (bar.h) with the following:

// android/some_header.h
#if __ANDROID_API__ >= 30
void some_api();
#elif defined(ANDROID_API_WEAK_AVAILABILITY)
void some_api() __attribute__((weak));
#endif

// bar.h
#include <android/some_header.h>

// foo.cpp
#include <bar.h>
#define ANDROID_API_WEAK_AVAILABILITY
#include <android/some_header.h>

void foo() {
    if (android_get_device_api_level() >= 30) {
         some_api();
    } else {
        fallback_impl();
    }
}

When this is build for minSdkVersion 29 or lower, foo.h will not compile because android/some_header.h will not be re-included with the new context because of the header guard. It also wouldn't be possible to separate these into android/some_header.h and android/some_header_weak.h because that would result in redefinition errors when both are included in the same TU.

In this particular example the problem can be fixed by including bar.h after android/some_header.h, but that's not actually a solution, it just reverses the problem in a way that happens to work for this example.

I don't think this is a particularly unrealistic issue given the C++ community's favor for header-only libraries.

As I said on the other thread, I think the better solution here is to make the jetpack libraries that handle the dlopen/dlsym shims for you. That's something we'll be doing either way. Still, not going to close this until we're sure.

[cpsauer]

Chiming in: This would indeed be super useful to me and my team.

Coming from iOS, I'd originally assumed that the NDK already had support for this as a basic backwards compatibility feature.

Their solution (support null checks+attributes, make library load weak, use available builtins, and let the compiler take care of enforcing the proper checks) is a really great developer experience.

[cpsauer]

If anyone has gotten this working already by patching the header, I'd love to hear from you! I can imagine ways you might be able to redefine __INTRODUCED_IN to get the desired behavior.

[ringoz]

Looks like the issue is fixed?
I am using __ANDROID_UNAVAILABLE_SYMBOLS_ARE_WEAK__ with great success in r23.

[DanAlbert]

I suppose we've technically done this because we added the option, but it's completely unsupported and untested. Until we're actually doing that I'm not comfortable even advertising the option (the option was added for a different use case than the NDK, with the expectation that it would also be used for the NDK once we had time to make it a tested configuration).

If you do end up running into issues later, please file them, but understand that they are inherently low priority as this isn't something we support yet.