Database Industry perspective
1 like261 views
This document discusses important database issues to consider before, during, and after large-scale system development. Before development, standards, thinking ahead about future needs, and managing different environments are key. During development, configuration management and ensuring adequate test data and processes for loading real data are essential. After deployment, planning for future changes, growth, and maintenance is important. The document emphasizes that advance planning, standards, and careful management are needed to develop systems that last and avoid errors.
Database Industry perspective
- 1. Database: An Industry PerspectiveLarge-scale system development is expensive. The resulting system needs to be a quality product and needs to last a long time. Some database issues to consider before, during, and after development: • Before – standards, thinking ahead, environments • During – configuration mgmnt, test data / data take-on • After – change, growth, maintenance Ref: Lake (2014)
- 2. CORE (a bespoke ERP type system)
- 3. Before • Develop and publish Standards – Sensible naming (short & meaningful) – Consistency (data typing, data formats, constraints) – Strategy (use of SP’s, triggers, views, de- normalisation, etc) – Procedures (reviews, testing, documentation) Important in large teams
- 4. Before • Thinking ahead Some issues might be . . . – Audit trail data – Business data value ranges ( + surrogate keys) – Currency flags / dates – Data owners – Use of indexes, partitioning (archiving ?) – Access control / management . . . but there may be others
- 5. Before • Development Environments – how many will you have ? – different schemes are possible – env. configuration management – specify all components, versions, and configuration parameters – must be able to recreate them – app. configuration management – plan promotion control of s/ware & d/b changes – plan test data population and refreshment – are there any integration partners – how will integration testing work ?
- 8. Different environment schemes aren’t the only issues if several systems are involved. Complexities may include: - different technologies / platforms - different organisations / people involved - different standards - different priorities - test data alignment Advance planning and scheduling at a detailed level is essential – lots of work for DBA’s.
- 10. During • Configuration management – keeping d/b changes and software versions in step – knowing what is where – keeping control • Test Data – will need lots of it, and it gets used up – need to plan where to get it from – and schedule population and refreshment • Data Take-on – don’t underestimate it !
- 12. Data Take On • Plan early Quantify and Plan early – preparation can be a long job ! 6 Months for Core- Proflow • Identify resources / sources / methods • Resources - Users and IT staff • Sources - may be documents, other systems, or may need to create new data • Methods - direct manual entry (via Application forms) - indirect manual entry (import from spreadsheets) - import from existing/other systems . . . last two need software, which needs to be written and tested, . . . enhanced and re-tested, etc
- 13. • Data preparation / cleaning May need reformatting, conversion to new schemes, missing data adding, errors correcting. Needs a lot of time (and knowledge) from USERS. • Loading Devise a schedule. Rehearse/Test it. How long does it take ? How large is the window ? • Validation Construct some comprehensive tests. Involve users. • Testing / Sign Off
- 15. After: Post-Implementation • People join or leave the company • Jobs and/or organisations change . . . impact on Access Control • Further developments • Integration with other systems . . . more Testing & Data Take-on • Business changes: some data expires • Database growth ? . . . reasons for thinking ahead, + h/w upgrades • Database Upgrades . . . always inconvenient, makes work, but essential
- 16. Access Control Which users can do what ? . . . Scheme needs to be • Secure Password protected, expiring regularly (35d). NOT having Groups (difficult though) • Workable Reasonable solution for current circumstances, in terms of people, jobs and roles. • Manageable Scheme capable of accommodating personnel and job changes in the future, without too much effort. • Managed Ideally managed by a business person, but often left to IT Dept. - responding to phone calls . . . ‘I need access to . . .’
- 18. • Both workable, but neither turned out very manageable in the long run . . . • Business reorganisations affect jobs and roles, so eventually no (db) roles fit. Easy to just issue extra roles with all functions intact, but it’s a security risk. • Total revamping is too time-consuming, and not a priority until something goes wrong. (financial auth. example) • Lots of different approaches, no right way. But most seem to use roles - because User -> Function is too much work.
- 19. An Industry Perspective Large-scale system development is expensive. The resulting system needs to be a quality product and needs to last a long time. Some database issues to consider before, during, and after development: • Before – standards, thinking ahead, environments • During – configuration mgmnt, test data / data take-on • After – change, growth, maintenance
- 20. Complex environments offer many opportunities for error. Which is why having established procedures and standards, thinking and planning ahead, having good configuration management, good communication, (and competent people) . . . is important. Otherwise, bad things can happen . . . • At a University not far away - Spurious updates to the Finance system - Traced to a redundant server + old copy of database. - Re-enabled for data access, but DBMS_JOB is still active, and posts old data to the live system . . . ! So be careful. Follow standard procedures, and get a second opinion !