occ_download with pred_not(pred_notnull("<>"))

[jhnwllr]

There is a way on GBIF to get NULL (empty values) by running a sort of hack (not not null).
example here: https://www.gbif.org/occurrence/download/0131893-200613084148143

I think this actually works with occ_download if you run something like this

occ_download_prep(
type="or",
pred_lt("coordinateUncertaintyInMeters",10000),
pred_not(pred_notnull("coordinateUncertaintyInMeters")),
format = "SIMPLE_CSV",
user=user,pwd=pwd,email=email
)

<<gbif download - prepared>>
Username: jwaller
E-mail: jwaller@gbif.org
Format: SIMPLE_CSV
Request:
type: or
predicates:
> type: lessThan, key: COORDINATE_UNCERTAINTY_IN_METERS, value(s): 10000
> type: not
predicate:

this also works

occ_download_prep(
type="or",
pred_lt("coordinateUncertaintyInMeters",10000),
pred_not(pred_notnull("coordinateUncertaintyInMeters")),
pred_and(
pred("basisOfRecord", "FOSSIL_SPECIMEN")
),
format = "SIMPLE_CSV",
user=user,pwd=pwd,email=email
)

However, it fails if I try to do it with pred_not...

occ_download_prep(
type="or",
pred_lt("coordinateUncertaintyInMeters",10000),
pred_not(pred_notnull("coordinateUncertaintyInMeters")),
pred_and(
pred_not(pred("basisOfRecord", "FOSSIL_SPECIMEN"))
),
format = "SIMPLE_CSV",
user=user,pwd=pwd,email=email
)

Error in pred_factory("=")(key, value) :
argument "value" is missing, with no default
Calls: occ_download_prep ... parse_predicates -> pred_and -> -> pred ->
Execution halted

Wondering if you know what is happening here? Thanks!

[sckott]

I can't replicate the problem. Can you share your session info?

[sckott]

I made some internal changes that aren't user facing, but also changed pred_or and pred_and so that user has to pass more than 1 predicate as as far as I know it doesn't make sense to use or/and predicates with only 1 predicate, correct? This change helps us avoid special handling I would have to add for something like

pred_and(
  pred_not(pred("basisOfRecord", "FOSSIL_SPECIMEN"))
)

where pred_and only receives one predicate

[jhnwllr]

Fails for me with pred_not(pred_notnull("coordinateUncertaintyInMeters")).

I would like this to work.

occ_download_prep(
type="and",
pred_or(
pred_lt("coordinateUncertaintyInMeters",10000),
pred_not(pred_notnull("coordinateUncertaintyInMeters"))
),
pred("taxonKey", 5052020),
pred("hasGeospatialIssue", FALSE),
pred("hasCoordinate", TRUE),
format = "SIMPLE_CSV",
user=user,pwd=pwd,email=email
)

Error in preds_factory("or")(.list, ...) :
1 or more inputs is not of class 'occ_predicate'; see docs
Calls: occ_download_prep -> parse_predicates -> pred_or ->

Works for me with pred_notnull("coordinateUncertaintyInMeters")

occ_download_prep(
type="and",
pred_or(
pred_lt("coordinateUncertaintyInMeters",10000),
pred_notnull("coordinateUncertaintyInMeters")
),
pred("taxonKey", 5052020),
pred("hasGeospatialIssue", FALSE),
pred("hasCoordinate", TRUE),
format = "SIMPLE_CSV",
user=user,pwd=pwd,email=email
)

<<gbif download - prepared>>
Username: jwaller
E-mail: jwaller@gbif.org
Format: SIMPLE_CSV
Request:
type: and
predicates:
> type: or
predicates:
- type: lessThan, key: COORDINATE_UNCERTAINTY_IN_METERS, value: 10000character(0)
> type: equals, key: TAXON_KEY, value(s): 5052020
> type: equals, key: HAS_GEOSPATIAL_ISSUE, value(s): false
> type: equals, key: HAS_COORDINATE, value(s): true

session info

R version 4.0.3 (2020-10-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages:
[1] stats graphics grDevices utils datasets methods base

other attached packages:
[1] dplyr_1.0.2 rgbif_3.5.1.91

loaded via a namespace (and not attached):
[1] Rcpp_1.0.5 whisker_0.4 xml2_1.3.2 magrittr_2.0.1
[5] tidyselect_1.1.0 munsell_0.5.0 uuid_0.1-4 colorspace_2.0-0
[9] R6_2.5.0 rlang_0.4.9 stringr_1.4.0 plyr_1.8.6
[13] httr_1.4.2 wk_0.3.4 tools_4.0.3 grid_4.0.3
[17] data.table_1.13.2 gtable_0.3.0 ellipsis_0.3.1 lazyeval_0.2.2
[21] tibble_3.0.4 lifecycle_0.2.0 crayon_1.3.4 purrr_0.3.4
[25] ggplot2_3.3.2 vctrs_0.3.5 glue_1.4.2 stringi_1.5.3
[29] oai_0.3.0 conditionz_0.1.0 compiler_4.0.3 pillar_1.4.7
[33] generics_0.1.0 scales_1.1.1 jsonlite_1.7.1 wellknown_0.7.2
[37] pkgconfig_2.0.3

[sckott]

Thanks! Woops, needed to fix a not predicate issue. Reinstall. This is what I'm getting

occ_download_prep(
      type="and",
      pred_or(
        pred_lt("coordinateUncertaintyInMeters",10000),
        pred_not(pred_notnull("coordinateUncertaintyInMeters"))
      ),
      pred("taxonKey", 5052020),
      pred("hasGeospatialIssue", FALSE),
      pred("hasCoordinate", TRUE),
      format = "SIMPLE_CSV"
    )
#> <<gbif download - prepared>>
#>   Username: sckott
#>   E-mail: myrmecocystus@gmail.com
#>   Format: SIMPLE_CSV
#>   Request:
#>     type:  and
#>     predicates:
#>       > type:  or
#>         predicates:
#>           - type: lessThan, key: COORDINATE_UNCERTAINTY_IN_METERS, value: 10000
#>           - type: not
#>             - type: isNotNull, parameter: COORDINATE_UNCERTAINTY_IN_METERS
#>       > type: equals, key: TAXON_KEY, value: 5052020
#>       > type: equals, key: HAS_GEOSPATIAL_ISSUE, value: false
#>       > type: equals, key: HAS_COORDINATE, value: true

[jhnwllr]

Yes thanks Scott! It works for me now on rgbif_3.5.1.92

[sckott]

great