DSpace Developers Workshop at OR2024.pdf

1
DSpace Developer Meet-Up and Q&A

2
Priorities for DSpace 8
IIIF Support and/or Basic
Image/Video viewer
DSpace 8.0 had a shortened development cycle and DSpace Steering established four priorities
Include major
features which just
missed the 7.x series
of releases
FEATURES INCLUDE
1
➢ COAR Notify
➢ OpenAIRE Correction service
➢ Port “REST-based Quality Control
Reports”
➢ Basic Duplicate Detection in
Submission form
New features which
empower users
(focus on the Admin
User Interface)
2
➢ Request withdrawal or
reinstatement of Items
➢ “Processes” page reorganization
➢ Edit Item supports authority control
lookup & lookup via external
sources
➢ Item Submission forms can be
configured for entire Community
➢
Any other features
which existed in 6.x
but missed the 7.x
series of releases
3
➢ Advanced Search
➢ Search facets on Homepage /
Community / Collection
➢ Search within Community /
Collection
Necessary bug fixes,
accessibility fixes,
performance
improvements &
dependency
upgrades
4
➢ Large number of accessibility fixes
(header, navbar, search, admin tools,
MyDSpace, etc.)
➢ Accessibility fixes backported to 7.6.2
(due in July/Aug)
➢ Major dependency upgrades:
○ Backend upgraded to Spring v6
○ Frontend upgraded to Angular 17
FEATURES INCLUDE FEATURES INCLUDE FEATURES INCLUDE

3
8.0 Statistics
IIIF Support and/or Basic
Image/Video viewer
>6,000 lines of code refactored for the dependency
upgrades
Testathon revealed a number of bugs which
were the result of these upgrades.
Community developers are still resolving
them.
DSpace 8.0 release: Approx. June 17-24

4
DSpace 8.0 Feature
Preview (Part 1)
OpenAIRE Data Correction & Publication Claim,
COAR Notify, and more…
4

5
● Content exists elsewhere: assure that your repository showcases all the production of your
community
● It is the result of the OpenAIRE ELD project awarded by 4Science back in 2020 and first
implemented in DSpace-CRIS in 2021 now finally available in DSpace 8
A new researcher joins the institution and logins for the first time in the repository.
The publication claim service found most of their publications and prompts for import.
The researcher reviews the list, confirms the authorship and imports
the publication saving a significant amount of time….
5
DSpace 8 Features - Publication Claim

6
6
DSpace 8 Features - OpenAIRE Publication Claim

7
7

8
8

9
9

10
Configurable entities
Researcher Profile
10
free icons from https://fontawesome.com/
Requires:

11
• It uses the generic service Suggestion Service to store,
retrieve and process suggestions related to content potential of
interest for researchers affiliated to the repository in SOLR (new
suggestion core)
• It is open to future extensions by mean of additional suggestion
providers (i.e. ORCID)
• The openAIRE provider includes a framework to evaluate the
quality of the suggestions (i.e. assign a score)
11

12
● Show reliable and accurate information in your repository
● It is the result of two innovative projects, the previous mentioned OpenAIRE ELD project and the
ELD ADVANCE project supported by the European Open Science Cloud Digital Innovation Hub
A repository manager of a repository indexed in OpenAIRE
login in the repository and see the list of improvements and
corrections identified, among others one publication that has a PMID that
was unknown to the repository and a link to a project.
They click on the “accept the suggestion” button and the new information
is stored in the local record.
12
DSpace 8 Features - OpenAIRE Data Correction

13
13

14
14

15
15

16
Your repository must be registered
as an OpenAIRE provider
Configurable entities
Researcher Profile
Quality Assurance
16
free icons from https://fontawesome.com/
Requires:

17
• It uses a generic service QAEventService to store, retrieve and
process corrections related to content already in the repository,
data are stored in SOLR (new qaevent core)
• It is open to future extensions by mean of additional correction
providers (see next slides)
• It includes a framework to automatically process corrections
according to configurable rules
17

18
18
DSpace 8 Features - DSpaceUsers Data Correction

19
19

20
• It is another provider of the Data Correction functionality just
presented, it generates QAEvent
• It collects correction proposals by the DSpace Users
(DSpaceUsers source), currently the following
CorrectionType(s) are provided:
• request to withdrawal
• request to reinstate
• Thanks to the previous mentioned processing framework it allows
automatic approval of request according to flexible rules (based
on the ItemFilter framework)
20

21
21
DSpace 8 Features - COAR Notify
I have a publication that uses your dataset
Link to a dataset is in the metadata
check if automatic
notification should
be triggered
LDNMessageConsumer
Link the dataset back to DSpace

22
22

23
23

24
24

25
25

26
26

27
27

28
28

29
29
DSpace 8 Features - ROR Integration

30
30
DSpace 8 Features - ROR Integration

31
DSpace 8.0 Feature
Preview (Part 2)
Improvements to Processes, embedded Search, Duplicate
Detection, Administrative Reports, and more…
31

32
Processes page redesigned
32

33
33

34
34

35
35

36
Basic Duplicate detection
Basic detection mechanism to prevent duplicate items in the repository.
● Compares item titles in SOLR with the submitted item’s title
● Configurable degree of variation (uses levenshtein distance)
Duplicates can be found:
● By submitters or editors via a submission step to warn submitters or editors of potential duplicates
● By Workflow reviewers or editors through a warning for claimed and pooled tasks indicating the total number of
potential duplicates.
● Through a new /duplicates REST item link which will search and retrieve a paged list of potential duplicates for
any item
36

37
37

38
38

39
Search tab on Community/Collection pages
39

40
Search tab on Community/Collection pages
40

41
Search Facets on Homepage
41

44
Add relationships via external sources
44

45
Administrator Reports (beta)
DSpace 8 will introduce basic administrator reports. There are two reports available:
● Filtered Collection reports: these reports count the numbers of item per community/collection. Administrators will
also have access to a number of filters they can apply to narrow down the result.
● Metadata Query reports: These will provide item counts based on metadata queries to specify what should be
counted and what not (and some other available filters).
45

46
Administrator Reports (beta)
46

47
Bug fixes
https://github.com/orgs/DSpace/projects/27/views/2?filterQuery=d
one+label%3Abug
47

48
Overview of Upgrade
process from v7 to v8
overview of backend major changes
48

49
Upgrading underlying DSpace 8 dependencies
• Java 17 (OpenJDK or Oracle)
• Apache Maven 3.8.x or above
• Tomcat is now OPTIONAL, if you want to use it you need version 10.
You may instead choose to use the Runnable JAR approach to deploying
the DSpace 8 backend.
java -jar [dspace]/webapps/server-boot.jar
• PostgreSQL 12.x - 15.x (with pgcrypto installed) - unchanged
• SOLR 8.11.1+ - unchanged
in bold the version currently used by the CI and dockerfile
49

50
How to update the server.xml with the embedded
Tomcat?
DSpace installation requires you to configure the URIEncoding=”UTF-8”
attribute over the Connector in the server.xml luckily this is not required
with the embedded tomcat as it is configured by default
Good default are provided also for the X-Forwared-For Header and to
trust internal proxies making available out-of-box a production ready
configuration.
If you need to customize other aspect of the server.xml (port, extra
connector, etc.) you can do that using env properties and/or modifying the
spring boot application.properties file
Spring boot reference documentation
50

51
Update your code base
• For upgrade from version prior of DSpace 7 there are some extra steps
required (older is your version more extra steps are needed) be sure to
look closely to the documentation
• if you are coming from a version prior of DSpace 7 consider to start with a
fresh codebase and manually port (reconsider) your changes
• if you are already on DSpace 7 you can merge or rebase your change on top
of DSpace 8 (there is a GitHub issue to improve the merge experience)
• include in the new local.cfg your changes or merge the change in the
configuration files with your, for example
db.dialect = org.dspace.util.DSpacePostgreSQLDialect
51

52
Major changes in the backend code
• JAVA 11 → 17:
• most javax.* dependencies
migrated to jakarta.*
• Hibernate 5 → 6:
• HibernateProxyHelper is now
in the org.dspace.core
package
• Query created via HQL/JPQL
have been updated to use
CriteriaBuilder
• Spring boot 2 → 3
• Spring framework 5 → 6
52
https://github.com/DSpace/DSpace/pull/9321

53
Major changes in SOLR
• There are no major changes 🙂 still 8.11.x
• Two new cores have been introduced: suggestion, qaevent. The
could contain data not available elsewhere (decision taken over
the proposed suggestion or correction) so it is a good idea to
include them in your backup strategy
53

54
Overview of Upgrade
process from v7 to v8
overview of frontend major changes
54

55
➔ 1. Merge & ﬁx conﬂicts
➔ 2. Upgrade node & install dependencies
➔ 3. Upgrade to standalone components
➔ 4. Run ESLint to apply the new rules
Upgrading the UI from 7 to 8 - Overview

56
Checkout the DSpace 8 tag from the community github
Merge in your DSpace 7 branch
1. Merge & fix conflicts
$ git checkout -b dspace-8-upgrade dspace-8.0.0
$ git merge prod

57
Why in that order?
➔ Ensures that your customizations show up as changes in the merge commit
➔ You’ll know why those changes are there
➔ You can re-evaluate if they are still necessary

58
➔ Fix any merge conflicts you might have with custom code
➔ For lists (e.g. of injected services in a constructor, or config properties etc), it’s good
practice to put the community changes on top, and yours at the bottom
➔ Leads to fewer merge conflicts down the line, because new community changes will
happen above your changes, not in between

59

60
Switch to node 18 or 20 if you haven’t already
Install the dependencies
2. Upgrade node & install dependencies
$ nvm install 20
$ nvm use 20
$ yarn run clean
$ yarn install

61

62
PR #2750 converted the UI to use standalone components:
➔ Every component speciﬁes its own dependencies
➔ They don’t need to be declared in a module
➔ Can be more easily reused in a different context
➔ Can be lazy-loaded separately, so better for performance
3. Upgrade to standalone components

63
Differences to the component decorator:
➔ Property standalone: true
➔ Optional list of imports
@Component({
selector: 'ds-base-login-page',
styleUrls: ['./login-page.component.scss'],
templateUrl: './login-page.component.html',
standalone: true,
imports: [ThemedLogInComponent, TranslateModule],
})
export class LoginPageComponent implements OnDestroy, OnInit {

64
More info on the wiki:
https://wiki.lyrasis.org/display/DSPACE/Migration+to+Standalone+Components

65
Convert your custom components to standalone. Path is optional
Choose the option: “Convert all components, directives and pipes to standalone”
? Choose the type of migration: (Use arrow keys)
❯ Convert all components, directives and pipes to standalone
Remove unnecessary NgModule classes
Bootstrap the application using standalone APIs
$ ng generate @angular/core:standalone --path src/themes/your-theme-name

66
The script will sometimes not add all required imports to the component decorator
Compare the list of imports in your themed components with the custom theme.
E.g. after the tool
3. Standalone components - Fix imports
@Component({
selector: 'ds-footer',
styleUrls: ['footer.component.scss'],
templateUrl: 'footer.component.html',
standalone: true,
imports: [NgIf, RouterLink]
})

67
The custom theme version:
So copy those imports to your version
@Component({
selector: 'ds-footer',
styleUrls: ['footer.component.scss'],
templateUrl: 'footer.component.html',
standalone: true,
imports: [NgIf, RouterLink, AsyncPipe, DatePipe, TranslateModule]
})

68
If you’ve added additional components to your templates and they haven’t been detected by
the script you’ll get errors like this, when you build:
The solution is then to add the corresponding component class to the imports:
Error:
src/themes/workshop/app/shared/object-list/community-list-element/community-list-element.component.html:
3:5 - error NG8001: 'ds-thumbnail' is not a known element:
1. If 'ds-thumbnail' is an Angular component, then verify that it is included in the
'@Component.imports' of this component.
2. If 'ds-thumbnail' is a Web Component then add 'CUSTOM_ELEMENTS_SCHEMA' to the '@Component.schemas' of
this component to suppress this message.
3 <ds-thumbnail [thumbnail]="(object.logo | async)?.payload">
imports: [NgIf, RouterLink, ThemedThumbnailComponent]

69
Then the next error is about the async pipe:
So add that as well
Error:
src/themes/workshop/app/shared/object-list/community-list-element/community-list-element.component.html:
3:47 - error NG8004: No pipe found with name 'async'.
3 <ds-thumbnail [thumbnail]="(object.logo | async)?.payload">
imports: [NgIf, RouterLink, ThemedThumbnailComponent, AsyncPipe]

70
Find the correct component class by ctrl/cmd clicking the HTML in your IDE
If that doesn’t work, search for “selector: 'ds-name-of-selector'“
@Component({
selector: 'ds-thumbnail,
styleUrls: [],
templateUrl: '../shared/theme-support/themed.component.html',
standalone: true,
imports: [ThumbnailComponent]
})
export class ThemedThumbnailComponent extends ThemedComponent<ThumbnailComponent>

71

72
PR #2343 added stricter ESlint rules
Goals:
➔ fewer merge conﬂicts
• Imports in alphabetical order
• multiple imports from same ﬁle on separate lines
• enforce trailing commas
4. ESLint - New rules

73
Goals:
➔ Remove ambiguity
• disallow !(value$ | async) in Angular templates
• disallow != and == in Angular templates
• disallow “harmful” types
• E.g. use string instead of String or () => {} instead of Function

74
Goals:
➔ Follow RxJS recommendations
• Don’t import from internal modules
• Instead of
Instead of
import { of as observableOf } from 'rxjs/internal/observable/of';
import { of as observableOf } from 'rxjs';

75
Goals:
➔ Follow RxJS recommendations
• Don’t nest subscribes
• Instead of
Use
obs$.pipe(
switchMap((v1) => this.someService.someMethod(v1)),
).subscribe((v2) => console.log(v2));
obs$.subscribe((v1) =>
this.someService.someMethod(v1).subscribe((v2) => console.log(v2))
);

76
Run lint-fix to automatically fix most issues
You may get a few errors that can’t be fixed automatically:
4. ESLint - Running ESLint
$ yarn run lint-fix --quiet
src/themes/workshop/app/shared/object-list/community-list-element/community-list-element.component.html
6:24 error Async pipe results should not be negated. @angular-eslint/template/no-negated-async
7:15 error Expected `!==` but received `!=` @angular-eslint/template/eqeqeq
7:54 error Expected `===` but received `==` @angular-eslint/template/eqeqeq

77
“Async pipe results should not be negated”
Reason: isLoading$ could be undefined, null or 0 too
Fix it by explicitly checking if it’s false
4. ESLint - Fix issues
<div *ngIf="!(isLoading$ | async)">
<div *ngIf="(isLoading$ | async) === false">

78
“Expected `!==` but received `!=`” or “Expected `===` but received `==`”
Reason: != and == follow complicated rules.
E.g. 100 == ‘100’ would be true
true == ‘true’ would be false
Fix it by using !== and ===
4. ESLint - Fix issues
<a *ngIf="linkType != linkTypes.None" [target]="(linkType == linkTypes.ExternalLink) ? ...
<a *ngIf="linkType !== linkTypes.None" [target]="(linkType === linkTypes.ExternalLink) ? ...

79
In DSpace 7, to make MyComponent themeable:
➔ Create ThemedComponent<MyComponent>
➔ Use selector ds-themed-my-component
➔ Replace all usages of ds-my-component with ds-themed-my-component
➔ Easy to forget a place it’s used
➔ When you forget, the app keeps working
➔ You’ll only notice when you theme it and realize the themed version isn’t used in a
certain context
4. ESLint - New themed component rules

80
In DSpace 8:
➔ Create ThemedComponent<MyComponent>
➔ But use the original selector ds-my-component
➔ Replace the selector of MyComponent with ds-base-my-component
➔ All existing usages now refer to the themed version automatically
➔ When using the component in a theme use ds-themed-my-component
• This is a convection: not strictly necessary
• The selector for a themed version of a component is never used directly

81
➔ PR #2865 adds an ESLint plugin to enforce these rules
➔ If you run yarn run lint-ﬁx it will automatically convert from the old style to the new one
becomes:
<ds-themed-thumbnail [thumbnail]="(object.logo | async)?.payload">
<ds-thumbnail [thumbnail]="(object.logo | async)?.payload">

82
becomes:
@Component({
selector: 'ds-themed-thumbnail,
...
})
@Component({
...
})

83
becomes:
@Component({
...
})
export class ThumbnailComponent implements OnChanges {
@Component({
selector: 'ds-base-thumbnail,
...
})
export class ThumbnailComponent implements OnChanges {

DSpace Developers Workshop at OR2024.pdf

More Related Content

Similar to DSpace Developers Workshop at OR2024.pdf (20)

More from 4Science (20)

Recently uploaded (20)

DSpace Developers Workshop at OR2024.pdf