SlideShare a Scribd company logo
Basic TF Query Language Power searching
Query Language Outline Formulating Queries Basics SEARCH and WHERE Regular Expression Basics Controlling Display Putting it all together
Query language A way of asking the datastore to extract a list nodes, and a subset of their attributes Performs the same function as SQL in a relational database It forms the basis of every datastore read Used internally by Atrium Discovery Used when viewing built-in reports Used for user-generated queries Used for ad-hoc user queries Used within TPL (patterns) There are special keywords to describe what to pick:  SEARCH, WHERE And keywords to decide how to display it:  SHOW, ORDER
Formulating Queries in the UI
Generic Query Interface Enter query, may split over several lines for clarity Press ‘Run Query’ to see results Results can be exported as CSV or XML
Previous Queries Under the search query window a list of previous queries is available Clicking these copies the query into the search box
Saving Previous Queries Use “//” to provide a title to your query Title is shown in Previous Queries  section Can drag and drop query to bookmark  Query will work on any Atrium Discovery instance
SEARCH and WHERE
SEARCH Looks through all the data and returns a set of nodes based on the specified node kind SEARCH  kind Find all nodes of given kind e.g. SEARCH SoftwareInstance These nodes are displayed as a list
Adding a WHERE SEARCH Host Collect the set of nodes of kind “Host” Add limiting conditions WHERE os_type = 'Solaris' Restrict the set of nodes to those that contain a specific string in the “os_type” attribute WHERE <attribute> [conditional statement]
Some Simple Conditions Equality a = b Not equals a <> b Similarly a > b a < b a >= b a <= b
More Advanced Conditions a HAS SUBSTRING b Works if the attribute contains the given string a HAS SUBWORD b Works if the attribute contains the given word a MATCHES b Lets you use pattern matching (Regular Expressions) a IN [b, c, d, e] Lets you search for a number of possible values
Multiple Conditions and Arithmetic Multiple conditions if you join them with logical operations AND OR NOT Brackets are useful to aid readability, and ensure precedence ( ) Integer arithmetic only + - * /
Special Conditions DEFINED This condition is true if the node has the attribute The value is not important Useful given the dynamic nature of the datastore SEARCH Host WHERE processor_type NOT DEFINED
A Word on Performance a = b a HAS SUBWORD b a HAS SUBSTRING b a MATCHES b Full use of datastore indexes Full use of datastore indexes Partial use of datastore indexes No use of datastore indexes SPEED
Alternatives to MATCHES Case insensitive search WHERE name MATCHES ‘(?i)fred’ (0.20s) WHERE name HAS SUBWORD ‘fred’ (0.03s) Multiple word match WHERE os MATCHES ‘(?i)Microsoft.+Datacenter’  (0.14s) WHERE os HAS SUBWORD 'Microsoft Datacenter‘ (0.09s) A list of possible values WHERE name MATCHES ‘(?i)TSL-D’ (0.24s) WHERE name IN ['tsl-dkirby','tsl-dtweed','tsl-duncan'] (0.08s)
Query Exercises
Exercise 1 SEARCH host  SEARCH Host SEARCH SoftwareInstance SEARCH DiscoveredFile Use “//” to title a report Drag and drop to create a bookmark
Exercise 2 SEARCH Host WHERE os = &quot;SunOS 5.10&quot;  SEARCH Host WHERE os <> &quot;SunOS 5.10&quot;  SEARCH Host WHERE name has substring &quot; Win &quot;   SEARCH Host WHERE name has subword &quot;Win&quot;
Exercise 3 SEARCH Host WHERE os = &quot;SunOS 5.10&quot; AND vendor = &quot;Sun Microsystems&quot; SEARCH Host WHERE os = &quot;SunOS 5.10&quot; AND vendor NOT matches &quot;Sun Microsystems&quot;
Regular Expression Basics
Character and Anchoring Matches Characters match themselves, except . ^ $ * + ? { } [ ] \ | () To match one of these you need to put a backslash before it Be careful with Foundation pathnames Use  \.   to match a full stop Use  \\  to match a (Windows) path separator Anchoring matches ^expr  – match only at start of string expr$  – match only at end of string
General Matches .   A period matches any character [ abcdef]  square brackets match any one of the things inside [^123]   match anything NOT contained within the brackets a|b  matches one thing or the other Brackets group things together:  (Fred)|(Fiona)  matches either name
Repeated Characters Very often want to repeat/extend matches to more than a single character Sometimes called wildcard matching ? matches 0 or 1 occurrences + matches 1 or more occurrences * matches 0 or more occurrences Control case matching (?i) means ignore case of matches
Exercise 4 SEARCH Host WHERE name matches '01app$' SEARCH Host WHERE name matches '02aPP$' SEARCH Host WHERE name matches '(?i)02aPP$'
Exercise 5 SEARCH Host WHERE name matches '(?i)0*App$' AND vendor matches '(?i)^sun' SEARCH Host WHERE name matches '(?i)0*App$' AND vendor NOT matches '(?i)^sun'
Controlling Display with SHOW
SHOW By default results set nodes are displayed by showing the “summary attributes” Defined in the taxonomy SHOW * All attributes defined in the taxonomy SHOW att1, att2 Only named attributes of the nodes SHOW summary, attr3 Summary attributes defined in the taxonomy, plus attr3 SHOW att1 AS ‘col1’ , att2 AS ‘col2’ Rename the column headings
Exercise 6 SEARCH Host WHERE name matches '(?i)0*App$' AND vendor NOT matches '(?i)^sun' SHOW * SEARCH Host WHERE name matches '(?i)0*App$' AND vendor NOT matches '(?i)^sun' SHOW hostname, os_type SEARCH Host WHERE name matches '(?i)0*App$' AND vendor matches '(?i)^sun'  SEARCH Host WHERE name matches '(?i)0*App$' AND vendor matches '(?i)^sun' SHOW summary, ram AS &quot;Memory&quot;
Putting it all together
Query Format Basic format of Query: SEARCH <kinds> [where clause] [show clause] <kinds> – Used to specify the nodes (objects) to search within [where clause] – Optional – filters the current set of nodes A WHERE clause evaluates an attribute WHERE hostname = “itapp” [show clause] – Optional – defines information to return SHOW summary, attribute1, attrbute2 SHOW hostname, os
Final Exercises
Exercise 7 For all hosts show following attributes: hostname, host type all os attributes Let’s refine the query: Limit the query to only windows hosts Show attributes a bit more interesting to windows hosts Add the virtual attribute Refine the query again: Limit by windows hosts that have any service pack details Refine the query once again: Limit by windows hosts that DO NOT have service pack details
Exercise 8 Refine the query from the last labs : To display all hosts except for windows Remove service pack limitation Remove the windows specific attributes Add attributes that are more interesting to non windows hosts Refine again by: Limit by hosts that have power supply status details Show package count totals
Online Documentation: http://www.tideway.com/confluence/display/81/Search+and+Reporting+Service http://docs.python.org/dev/howto/regex.html http://www.tideway.com/configipedia/Writing_Efficient_Regex Further Resources

More Related Content

ODP
Intermediate Perl
ODP
Introduction to Perl
ODP
Introduction to Perl - Day 2
PPT
LPW: Beginners Perl
PDF
Perl 5.10 for People Who Aren't Totally Insane
ODP
Advanced Perl Techniques
KEY
Introduction to Perl Best Practices
Intermediate Perl
Introduction to Perl
Introduction to Perl - Day 2
LPW: Beginners Perl
Perl 5.10 for People Who Aren't Totally Insane
Advanced Perl Techniques
Introduction to Perl Best Practices

What's hot (20)

PDF
Perl Scripting
PPT
Perl Presentation
PDF
Perl programming language
ODP
Introduction to Perl - Day 1
ODP
Introduction to Modern Perl
PPT
Ods Markup And Tagsets: A Tutorial
PPT
Perl 101 - The Basics of Perl Programming
PPT
Exploiting Php With Php
PPT
Power Theming
PDF
Working with text, Regular expressions
PDF
Improving Dev Assistant
PDF
Introduction to Perl and BioPerl
ODP
Beginning Perl
PPT
Bioinformatica 10-11-2011-p6-bioperl
PPTX
Bioinformatics p1-perl-introduction v2013
ODP
Introducing Modern Perl
KEY
Building and Distributing PostgreSQL Extensions Without Learning C
Perl Scripting
Perl Presentation
Perl programming language
Introduction to Perl - Day 1
Introduction to Modern Perl
Ods Markup And Tagsets: A Tutorial
Perl 101 - The Basics of Perl Programming
Exploiting Php With Php
Power Theming
Working with text, Regular expressions
Improving Dev Assistant
Introduction to Perl and BioPerl
Beginning Perl
Bioinformatica 10-11-2011-p6-bioperl
Bioinformatics p1-perl-introduction v2013
Introducing Modern Perl
Building and Distributing PostgreSQL Extensions Without Learning C
Ad

Viewers also liked (6)

PPT
Linq
PPTX
LINQ for absolute beginners
KEY
Introducing LINQ
PPT
Module 3: Introduction to LINQ (PowerPoint Slides)
PPT
Of Lambdas and LINQ
Linq
LINQ for absolute beginners
Introducing LINQ
Module 3: Introduction to LINQ (PowerPoint Slides)
Of Lambdas and LINQ
Ad

Similar to Addmi 10.5-basic query-language (20)

PDF
VMworld 2013: Deep Dive into vSphere Log Management with vCenter Log Insight
PPTX
PostgreSQL - It's kind've a nifty database
PDF
Bollean Search - NageshRao
PDF
Postgres performance for humans
KEY
Searching ORM: First Why, Then How
PDF
JDD 2016 - Tomasz Borek - DB for next project? Why, Postgres, of course
PPTX
Searching in AtoM
PDF
Hypertable - massively scalable nosql database
PDF
PPT
Intro to Data warehousing lecture 11
PPT
Intro to Data warehousing lecture 14
PPT
Intro to Data warehousing lecture 19
DOC
Boolean operators
PPTX
Splunk bangalore user group 2020 08 01
PPT
Slides
PDF
Beyond php - it's not (just) about the code
PDF
Mysql query optimization
PPTX
Postgres indexes
PDF
MAKE SENSE OF YOUR BIG DATA
PPTX
Unifying your data management with Hadoop
VMworld 2013: Deep Dive into vSphere Log Management with vCenter Log Insight
PostgreSQL - It's kind've a nifty database
Bollean Search - NageshRao
Postgres performance for humans
Searching ORM: First Why, Then How
JDD 2016 - Tomasz Borek - DB for next project? Why, Postgres, of course
Searching in AtoM
Hypertable - massively scalable nosql database
Intro to Data warehousing lecture 11
Intro to Data warehousing lecture 14
Intro to Data warehousing lecture 19
Boolean operators
Splunk bangalore user group 2020 08 01
Slides
Beyond php - it's not (just) about the code
Mysql query optimization
Postgres indexes
MAKE SENSE OF YOUR BIG DATA
Unifying your data management with Hadoop

More from odanyboy (20)

PPT
Addmi 18-appliance baseline
PPT
Addmi 17-snapshot
PPT
Addmi 16-discovery monitoring
PPT
Addmi 16.5-discovery troubleshooting
PPT
Addmi 15-discovery scripts
PPT
Addmi 14-discovery credentials
PPT
Addmi 13-discovery overview
PPT
Addmi 13-discovery overview (patrick ryan's conflicted copy 2011-01-27)
PPT
Addmi 12-basic scan
PPT
Addmi 11-intro to-patterns
PPT
Addmi 10-query builder
PPT
Addmi 09-analysis ui-reporting
PPT
Addmi 09.5-analysis ui-host-grouping
PPT
Addmi 08-dashboards
PPT
Addmi 07-taxonomy
PPT
Addmi 06-security mgmt
PPT
Addmi 04-va installation
PPT
Addmi 03-addm prerequisites
PPT
Addmi 02-addm overview
PPT
Addmi 01-why addm
Addmi 18-appliance baseline
Addmi 17-snapshot
Addmi 16-discovery monitoring
Addmi 16.5-discovery troubleshooting
Addmi 15-discovery scripts
Addmi 14-discovery credentials
Addmi 13-discovery overview
Addmi 13-discovery overview (patrick ryan's conflicted copy 2011-01-27)
Addmi 12-basic scan
Addmi 11-intro to-patterns
Addmi 10-query builder
Addmi 09-analysis ui-reporting
Addmi 09.5-analysis ui-host-grouping
Addmi 08-dashboards
Addmi 07-taxonomy
Addmi 06-security mgmt
Addmi 04-va installation
Addmi 03-addm prerequisites
Addmi 02-addm overview
Addmi 01-why addm

Recently uploaded (20)

PPTX
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
PDF
Encapsulation_ Review paper, used for researhc scholars
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
PPTX
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
PDF
Advanced methodologies resolving dimensionality complications for autism neur...
DOCX
The AUB Centre for AI in Media Proposal.docx
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
PPT
Teaching material agriculture food technology
PDF
Agricultural_Statistics_at_a_Glance_2022_0.pdf
PDF
Approach and Philosophy of On baking technology
PPTX
MYSQL Presentation for SQL database connectivity
PPTX
sap open course for s4hana steps from ECC to s4
PDF
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
PDF
KodekX | Application Modernization Development
PDF
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
PDF
MIND Revenue Release Quarter 2 2025 Press Release
PDF
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
PDF
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...
ACSFv1EN-58255 AWS Academy Cloud Security Foundations.pptx
Encapsulation_ Review paper, used for researhc scholars
Reach Out and Touch Someone: Haptics and Empathic Computing
VMware vSphere Foundation How to Sell Presentation-Ver1.4-2-14-2024.pptx
Advanced methodologies resolving dimensionality complications for autism neur...
The AUB Centre for AI in Media Proposal.docx
20250228 LYD VKU AI Blended-Learning.pptx
Teaching material agriculture food technology
Agricultural_Statistics_at_a_Glance_2022_0.pdf
Approach and Philosophy of On baking technology
MYSQL Presentation for SQL database connectivity
sap open course for s4hana steps from ECC to s4
Architecting across the Boundaries of two Complex Domains - Healthcare & Tech...
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
KodekX | Application Modernization Development
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
MIND Revenue Release Quarter 2 2025 Press Release
Profit Center Accounting in SAP S/4HANA, S4F28 Col11
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
TokAI - TikTok AI Agent : The First AI Application That Analyzes 10,000+ Vira...

Addmi 10.5-basic query-language

  • 1. Basic TF Query Language Power searching
  • 2. Query Language Outline Formulating Queries Basics SEARCH and WHERE Regular Expression Basics Controlling Display Putting it all together
  • 3. Query language A way of asking the datastore to extract a list nodes, and a subset of their attributes Performs the same function as SQL in a relational database It forms the basis of every datastore read Used internally by Atrium Discovery Used when viewing built-in reports Used for user-generated queries Used for ad-hoc user queries Used within TPL (patterns) There are special keywords to describe what to pick: SEARCH, WHERE And keywords to decide how to display it: SHOW, ORDER
  • 5. Generic Query Interface Enter query, may split over several lines for clarity Press ‘Run Query’ to see results Results can be exported as CSV or XML
  • 6. Previous Queries Under the search query window a list of previous queries is available Clicking these copies the query into the search box
  • 7. Saving Previous Queries Use “//” to provide a title to your query Title is shown in Previous Queries section Can drag and drop query to bookmark Query will work on any Atrium Discovery instance
  • 9. SEARCH Looks through all the data and returns a set of nodes based on the specified node kind SEARCH kind Find all nodes of given kind e.g. SEARCH SoftwareInstance These nodes are displayed as a list
  • 10. Adding a WHERE SEARCH Host Collect the set of nodes of kind “Host” Add limiting conditions WHERE os_type = 'Solaris' Restrict the set of nodes to those that contain a specific string in the “os_type” attribute WHERE <attribute> [conditional statement]
  • 11. Some Simple Conditions Equality a = b Not equals a <> b Similarly a > b a < b a >= b a <= b
  • 12. More Advanced Conditions a HAS SUBSTRING b Works if the attribute contains the given string a HAS SUBWORD b Works if the attribute contains the given word a MATCHES b Lets you use pattern matching (Regular Expressions) a IN [b, c, d, e] Lets you search for a number of possible values
  • 13. Multiple Conditions and Arithmetic Multiple conditions if you join them with logical operations AND OR NOT Brackets are useful to aid readability, and ensure precedence ( ) Integer arithmetic only + - * /
  • 14. Special Conditions DEFINED This condition is true if the node has the attribute The value is not important Useful given the dynamic nature of the datastore SEARCH Host WHERE processor_type NOT DEFINED
  • 15. A Word on Performance a = b a HAS SUBWORD b a HAS SUBSTRING b a MATCHES b Full use of datastore indexes Full use of datastore indexes Partial use of datastore indexes No use of datastore indexes SPEED
  • 16. Alternatives to MATCHES Case insensitive search WHERE name MATCHES ‘(?i)fred’ (0.20s) WHERE name HAS SUBWORD ‘fred’ (0.03s) Multiple word match WHERE os MATCHES ‘(?i)Microsoft.+Datacenter’ (0.14s) WHERE os HAS SUBWORD 'Microsoft Datacenter‘ (0.09s) A list of possible values WHERE name MATCHES ‘(?i)TSL-D’ (0.24s) WHERE name IN ['tsl-dkirby','tsl-dtweed','tsl-duncan'] (0.08s)
  • 18. Exercise 1 SEARCH host SEARCH Host SEARCH SoftwareInstance SEARCH DiscoveredFile Use “//” to title a report Drag and drop to create a bookmark
  • 19. Exercise 2 SEARCH Host WHERE os = &quot;SunOS 5.10&quot; SEARCH Host WHERE os <> &quot;SunOS 5.10&quot; SEARCH Host WHERE name has substring &quot; Win &quot; SEARCH Host WHERE name has subword &quot;Win&quot;
  • 20. Exercise 3 SEARCH Host WHERE os = &quot;SunOS 5.10&quot; AND vendor = &quot;Sun Microsystems&quot; SEARCH Host WHERE os = &quot;SunOS 5.10&quot; AND vendor NOT matches &quot;Sun Microsystems&quot;
  • 22. Character and Anchoring Matches Characters match themselves, except . ^ $ * + ? { } [ ] \ | () To match one of these you need to put a backslash before it Be careful with Foundation pathnames Use \. to match a full stop Use \\ to match a (Windows) path separator Anchoring matches ^expr – match only at start of string expr$ – match only at end of string
  • 23. General Matches . A period matches any character [ abcdef] square brackets match any one of the things inside [^123] match anything NOT contained within the brackets a|b matches one thing or the other Brackets group things together: (Fred)|(Fiona) matches either name
  • 24. Repeated Characters Very often want to repeat/extend matches to more than a single character Sometimes called wildcard matching ? matches 0 or 1 occurrences + matches 1 or more occurrences * matches 0 or more occurrences Control case matching (?i) means ignore case of matches
  • 25. Exercise 4 SEARCH Host WHERE name matches '01app$' SEARCH Host WHERE name matches '02aPP$' SEARCH Host WHERE name matches '(?i)02aPP$'
  • 26. Exercise 5 SEARCH Host WHERE name matches '(?i)0*App$' AND vendor matches '(?i)^sun' SEARCH Host WHERE name matches '(?i)0*App$' AND vendor NOT matches '(?i)^sun'
  • 28. SHOW By default results set nodes are displayed by showing the “summary attributes” Defined in the taxonomy SHOW * All attributes defined in the taxonomy SHOW att1, att2 Only named attributes of the nodes SHOW summary, attr3 Summary attributes defined in the taxonomy, plus attr3 SHOW att1 AS ‘col1’ , att2 AS ‘col2’ Rename the column headings
  • 29. Exercise 6 SEARCH Host WHERE name matches '(?i)0*App$' AND vendor NOT matches '(?i)^sun' SHOW * SEARCH Host WHERE name matches '(?i)0*App$' AND vendor NOT matches '(?i)^sun' SHOW hostname, os_type SEARCH Host WHERE name matches '(?i)0*App$' AND vendor matches '(?i)^sun' SEARCH Host WHERE name matches '(?i)0*App$' AND vendor matches '(?i)^sun' SHOW summary, ram AS &quot;Memory&quot;
  • 30. Putting it all together
  • 31. Query Format Basic format of Query: SEARCH <kinds> [where clause] [show clause] <kinds> – Used to specify the nodes (objects) to search within [where clause] – Optional – filters the current set of nodes A WHERE clause evaluates an attribute WHERE hostname = “itapp” [show clause] – Optional – defines information to return SHOW summary, attribute1, attrbute2 SHOW hostname, os
  • 33. Exercise 7 For all hosts show following attributes: hostname, host type all os attributes Let’s refine the query: Limit the query to only windows hosts Show attributes a bit more interesting to windows hosts Add the virtual attribute Refine the query again: Limit by windows hosts that have any service pack details Refine the query once again: Limit by windows hosts that DO NOT have service pack details
  • 34. Exercise 8 Refine the query from the last labs : To display all hosts except for windows Remove service pack limitation Remove the windows specific attributes Add attributes that are more interesting to non windows hosts Refine again by: Limit by hosts that have power supply status details Show package count totals
  • 35. Online Documentation: http://www.tideway.com/confluence/display/81/Search+and+Reporting+Service http://docs.python.org/dev/howto/regex.html http://www.tideway.com/configipedia/Writing_Efficient_Regex Further Resources

Editor's Notes

  • #15: The search is useful to find Hosts where we have not been able to determine the CPU type.
  • #19: Note: “search host” is of course meant to fail.
  • #20: Note: the last query is meant to fail.
  • #26: Note: The second query is meant to fail.