Thirty-three invented bookshops in real
British towns, with real coordinates, a genre taxonomy, an
author-influence graph and a walking route between the shops. It's
built to get a beginner to property paths and nested aggregation
quickly — and it's deliberately awkward in the places where SPARQL
is genuinely hard.
4,826triples, RDF 1.1
145worked queries
3engines, all verified
33bookshops in 26 towns
74works, 5 languages
What you're querying
The places are real. Real names, real WGS84
coordinates, real British National Grid eastings and northings — so
distances are checkable against a map and the geography module is
about actual geography. Everything else is invented.
Shops, people, publishers, books, events, prices, disputed claims. No
query here can teach you a false fact about a real person or
business.
Where to run it
The course is built around the Turtle Editor Viewer: a
browser editor that parses your Turtle, draws the graph and runs
SPARQL in the same window. Load data/04-bookshops.ttl,
look at the graph, then query it.
Modules 01–09, 11 and 12 also run unchanged in HOLOS and
Fuseki. Module 10 needs GeoSPARQL functions, so in practice
it needs HOLOS.
The awkward bits, on purpose
The place hierarchy is uneven — England has a region level,
Scotland and Wales don't — so a three-hop chain silently misses
two countries.
Two shops connect to nothing else. Books before 1970 have no
ISBN. Four founding dates are disputed by rival sources. Each gap
exists so a lesson has something real to find.
What the cross-engine checking found
Every query was run on all three engines and the answers compared
value by value, not by row count — which hides real disagreements.
Some of what that turned up is worth knowing before you write
anything of your own.
Written as
Editor (Comunica)
HOLOS
Fuseki (ARQ)
xsd:integer(?gYear)
0 rows, no error
0 rows, no error
works
xsd:integer(STR(?gYear))
works
works
works
{| ... |} annotation pattern
works
works
works
<<( ?s ?p ?o )>> triple term
works
works
works
<< s p o ~ ?r >> in a query
parse error
parse error
parse error
isTRIPLE, SUBJECT, LANGDIR
works
works
works
VERSION()
parse error
parse error
works
geof: functions
none at all
45 of them
warns, returns the row, leaves the value unbound
The first row is the dangerous one. Casting an xsd:gYear
straight to an integer returns zero rows on two of the three
engines, and raises no error anywhere — it looks exactly like a fact
about your data. Go via STR(). Q07 is built around it.
The shape of the data
Five classes carry most of the course. The three curved edges are the
recursive ones — a place inside a place, an author who read an author,
a genre under a genre — and they are where module 05 lives.
Module 00
The lab
Before the first query, learn the room. The Turtle
Editor Viewer does four things a plain SPARQL endpoint doesn't: it draws
the graph you're querying, reasons over it, converts it between formats
and validates it against SHACL — all in the browser, with nothing
installed — the editor is used online, at
semantechs.co.uk/turtle-editor-viewer.
These aren't SPARQL exercises. They take twenty minutes and they make
every query afterwards easier to picture.
00.1
Load something small and look at it
Open data/04-bookshops.ttl with Choose File,
and spend five minutes in the graph pane before writing anything.
How it works
The editor parses as you type, finds the subjects and draws the
first ten of them. Ten is the cap, and it's exactly why this course
ships eleven small module files rather than one large one: the
combined dataset is perfectly good to query and almost
useless to look at.
What to take away
Show labels, Node Labels and Property Labels
are ticked when the editor opens and show labels where the IRIs
would be. Untick them to see the IRIs a query has to use; with
Show labels on, the shop below is listed as The Inkwell.
Subjects dropdown — pick bt:shop-inkwell
on its own. That node and its dozen edges are precisely what Q02
returns as a table.
Engine: dot, then neato, then circo. The same graph, three
different questions answered — hierarchy, clustering, ring
structure.
Hide Types and Hide Annotations strip the class and
label edges, leaving the skeleton that module 05 walks.
Get All re-reads the pane and redraws from everything in
it — and, importantly, reloads the internal triplestore
the SPARQL panel queries. Edit the Turtle, press Get All, and
only then does your query see the change.
Do this one properly:
load data/03-places.ttl
set Hide Types, Hide Annotations
engine dot
You are now looking at the containment hierarchy that
module 05 spends ten queries on. Notice that the
English branch runs one level deeper than the Scottish
and Welsh ones:
york -> north-yorkshire -> yorkshire -> england
edinburgh -> edinburgh-city -> scotland
That asymmetry is the entire point of Q28, and it is
visible here before you write a line of SPARQL.
00.2
Watch a reasoner do what a property path does
Click Show Facts. HyLAR, an OWL 2 RL reasoner, runs in the
browser and shows you what it worked out.
How it works
The vocabulary is written to give it real work:
bs:within is declared an
owl:TransitiveProperty, bs:connectsTo an
owl:SymmetricProperty, and bs:hasImprint is
the owl:inverseOf bs:imprintOf — a property
asserted nowhere in the data at all.
Load data/03-places.ttl and press Show Facts. Because
containment is transitive, the reasoner materialises York to
Yorkshire, York to England and York to Great Britain as real triples,
which you can then match with a plain one-hop pattern.
What to take away
That's the same answer bs:within+ computes, reached
from the opposite end.
Neither is right in general. Seeing both before module 05 is what
makes property paths feel like a choice rather than the only
option.
Try it on data/05-people.ttl as well: Q44 builds the
identical inverse triples with CONSTRUCT instead.
reasoner property path
-------- -------------
work happens once, up front every time you ask
costs storage, and a nothing until asked
re-run on change
you then write ?t bs:within ?a ?t bs:within+ ?a
portability needs a reasoner any SPARQL 1.1 engine
Two routes to the same set of triples. This course
takes the second, because it travels.
00.3
Convert, validate, and share by URL
The remaining editor features worth knowing before you
start on the queries.
What to take away
Add Prefixes reads the prefixes out of the loaded data and
prepends them to your query. The course's queries already carry
the few they need, so this is for when you write your own.
Get All, again. Worth saying twice: an edit you have not
pressed Get All after is invisible to the SPARQL panel, and the
stale answer looks exactly like a wrong query. First thing to
check when a change appears to do nothing.
To JSON-LD / To Turtle round-trip the data. Worth doing
once with data/10-annotations-1.2.ttl open: RDF 1.2
triple terms have no settled JSON-LD form, so the conversion is
where you discover what your toolchain actually supports. Better
here than in a pipeline.
SHACL. The panel at the bottom right validates as well as
queries. Open the shapes in a second tab (the + on the tab
strip, then Load URL), pick that tab in the Shapes dropdown,
switch back to the data and press Validate. On the untouched
data shapes.ttl reports eight violations, all
from one shape: sh:lessThan between two
xsd:gYear values. SPARQL's < is not
defined for gYear, and the editor's engine treats a comparison it
cannot make as a failure, where the HOLOS command line compares the
years and reports clean. q07 is the same fact seen from a query.
Anything else it reports is real: set a staff count to zero and
validate again.
Load by URL. The toolbar takes a URL, and the app accepts
?dot=<url> for data and
&shapes=<url> for a shapes file, which opens
in its own tab already selected for validation. A link can carry a
whole exercise.
A CONSTRUCT is the loop back to the picture:
the full dataset
|
| Q47, a summary CONSTRUCT
v
~165 triples of Turtle
|
| copy out of the results pane
v
paste into the editor pane
|
v
the graph view draws that instead
The editor's ten-subject cap stops being a limit the
moment you get to choose which ten.
Module 01
First queries
Everything here runs in the browser. Paste the data into the Turtle Editor Viewer, paste the query into the SPARQL panel, press Execute. The aim of this module is to make the shape of a query familiar: a graph pattern goes in, a table comes out.
# Q01 Every bookshop on the trail
# What bookshops are in this dataset, and what are they called?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?shop ?name
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name .
}
ORDER BY ?name
One triple pattern finds the shops, a second fetches each one's label. Both patterns mention ?shop, so the engine keeps only the combinations where the same ?shop satisfies both -- that shared variable is the join, and it's the only join mechanism SPARQL has.
What to take away
A query is a graph pattern: you draw the shape you want and the engine finds every place the shape fits.
`a` is shorthand for rdf:type. It's the one abbreviation SPARQL borrows from Turtle.
Repeating a variable joins the patterns. There's no JOIN keyword because there doesn't need to be one.
?shop ---- rdf:type ----> bs:Bookshop (which things are shops)
|
+------- rdfs:label ---> ?name (what each is called)
Both lines constrain the SAME ?shop, so a row survives only if
both are true of it. 33 shops in, 33 rows out.
Editor33HOLOS33Fuseki33bookshop-trail-1.1.ttl
Q02
Everything known about one shop
What does the dataset actually record about The Inkwell?
# Q02 Everything known about one shop
# What does the dataset actually record about The Inkwell?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
SELECT ?p ?o
WHERE {
bt:shop-inkwell ?p ?o .
}
ORDER BY ?p
Fixing the subject and leaving predicate and object as variables returns every statement with that subject. It's the fastest way to find out how a strange dataset is shaped, and worth doing before writing anything more ambitious.
What to take away
Any position of a triple pattern can be a variable, including the predicate.
This is the single most useful exploratory query there's: run it against one resource before you try to query a thousand.
In the Turtle Editor Viewer you can do the same thing visually -- pick the shop in the Subjects dropdown and look at the graph.
bt:shop-inkwell ---- ?p ----> ?o
known unknown
(the subject) (everything else)
Turn the pattern inside out and you learn the vocabulary:
?p = bs:founded ?o = 1979
?p = bs:locatedIn ?o = bt:place-wigtown
?p = bs:hasCafe ?o = true ... and so on
Editor20HOLOS20Fuseki20bookshop-trail-1.1.ttl
Q03
Which town is each shop in
Pair every shop with the name of the town it trades in.
# Q03 Which town is each shop in
# Pair every shop with the name of the town it trades in.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?shopName ?townName
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?shopName ;
bs:locatedIn ?town .
?town rdfs:label ?townName .
}
ORDER BY ?townName ?shopName
Three patterns chained through two join variables. ?shop links the first two, ?town links the second and third. The engine is free to evaluate them in any order it likes; what matters to you is only that the shared variables line up.
What to take away
Chaining patterns walks the graph one edge at a time.
Semicolon repeats the subject; a full stop starts a new one. This is Turtle syntax reused inside SPARQL.
Labels live on the thing, not on the link. To show a name you almost always need one extra pattern per resource.
?shop --rdfs:label--> ?shopName
|
+--bs:locatedIn--> ?town --rdfs:label--> ?townName
Two joins: ?shop ties lines 1 and 2
?town ties lines 2 and 3
A chain like this is how you walk one hop at a time. Module 05
replaces the whole chain with a single path expression.
# Q04 The ten oldest shops
# Which shops have been trading longest?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?name ?founded
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name ;
bs:founded ?founded .
}
ORDER BY ?founded
LIMIT 10
ORDER BY sorts the result table after the pattern has matched, and LIMIT truncates it. Both act on the finished table, not on the matching -- so the engine still had to consider all 33 shops to know which ten come first.
What to take away
ORDER BY and LIMIT are applied to the result table, after matching.
ORDER BY ?x is ascending; wrap it as DESC(?x) for the other direction.
LIMIT without ORDER BY gives you an arbitrary ten rows, not the first ten of anything. The two belong together.
match -> 33 rows -> ORDER BY ?founded -> LIMIT 10 -> 10 rows
(ascending)
Order of evaluation, which is NOT the order you write them in:
WHERE -> GROUP BY -> HAVING -> ORDER BY -> OFFSET/LIMIT
^
SELECT's projection happens around here
# Q126 Reading a long answer ten at a time
# Show shops 11 to 20 of the trail, alphabetically.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?name
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name .
}
ORDER BY ?name
LIMIT 10
OFFSET 10
LIMIT caps the number of rows; OFFSET says how many to skip first. Together they page. The part everyone gets wrong is that neither means anything without ORDER BY: with no order, the second page may repeat rows from the first, and no engine is doing anything wrong.
What to take away
LIMIT takes n rows; OFFSET skips n first. Neither is meaningful without ORDER BY.
Order by something unique, or a page boundary inside a group of tied rows will drop and repeat.
OFFSET n makes the engine produce and discard n rows. Deep paging is expensive everywhere.
ORDER BY ?name 33 shops, in a defined order
LIMIT 10 take ten
OFFSET 10 after skipping ten
page 1 OFFSET 0 Bookbarrow ... Errata
page 2 OFFSET 10 Ex Libris ... The Bookwyrm <- this
page 3 OFFSET 20 The Broads Bindery ... The Sea Margin
page 4 OFFSET 30 Turn the Page ... West Quay Books
Thirty-three shops, so the last page has three rows rather
than ten. A client that stops when a page comes back short is
doing the right thing; one that stops when a page comes back
empty makes one extra request and is also fine.
without ORDER BY:
+----------------------------------------------------------+
| LIMIT 10 OFFSET 10 is "ten rows from somewhere in the |
| middle of an arbitrary sequence". Run it twice and you |
| may get different rows. Page through a whole table that |
| way and you will miss some and see others twice. |
+----------------------------------------------------------+
Order by something unique, too. Ordering by a column with ties
leaves the tied rows in an arbitrary order between them, and a
page boundary landing inside a tie has the same problem in
miniature -- q41 hit exactly this.
Paging costs the engine the whole sorted answer every time, so
page 400 is as expensive as pages 1 to 400 together. For a
large result set that is the thing to design around, not the
LIMIT.
Editor10HOLOS10Fuseki10bookshop-trail-1.1.ttl
Q05
What kinds of thing are in here
Without knowing anything about the dataset, what classes does it contain?
# Q05 What kinds of thing are in here
# Without knowing anything about the dataset, what classes does it contain?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
SELECT DISTINCT ?type
WHERE {
?s a ?type .
}
ORDER BY ?type
Match every typed resource, project only the type, and ask for DISTINCT. Anything that survives is a class the data actually uses -- which isn't always the same as the classes the schema declares.
What to take away
DISTINCT removes duplicate rows from the result table.
Querying the data for its own structure beats trusting the documentation, because the data can't be out of date with itself.
The classes you get back are the ones in use. A schema may declare more.
?s --rdf:type--> ?type
every typed thing collapse duplicates
(about 1,080 rows) --> DISTINCT --> ~20 classes
Ask this first. Then ask q06 to find out what properties each
class carries. Two queries and you have the map.
# Q06 What can I ask about a bookshop
# Which properties do bookshops actually carry?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT DISTINCT ?property ?comment
WHERE {
?shop a bs:Bookshop ;
?property ?value .
OPTIONAL { ?property rdfs:comment ?comment . }
}
ORDER BY ?property
Restrict to subjects that are bookshops, then leave the predicate free. The result is the working vocabulary for that class -- the list of things it's worth asking a shop about.
What to take away
Constraining the subject and freeing the predicate profiles a class.
This dataset carries its own schema, so a query can fetch the human-readable comment alongside the property name.
OPTIONAL is used here because a property isn't guaranteed to have a comment. Module 03 explains why that matters.
?shop --rdf:type--> bs:Bookshop (restrict the subject...)
|
+----- ?p ------> ?o (...then free the predicate)
Pair this with the schema itself:
bs:founded rdfs:comment "The year the shop opened."
The dataset describes its own vocabulary in 01-vocabulary.ttl, so
you can join the two and get documentation in your result table.
Editor16HOLOS16Fuseki16bookshop-trail-1.1.ttl
Module 02
Filtering and expressions
A pattern says which shape to match; a FILTER says which of the matches to keep. This module is also where the built-in functions live, and where the difference between a value and its lexical form starts to matter.
# Q07 Shops founded before 1970
# Which shops predate 1970, and why is the obvious way to ask wrong?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
SELECT ?name ?founded
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name ;
bs:founded ?founded .
FILTER( xsd:integer(STR(?founded)) < 1970 )
}
ORDER BY ?founded
FILTER drops every solution its expression doesn't judge true. The interesting part is the cast. bs:founded is an xsd:gYear, which is not a number, so it has to be converted before it can be compared with 1970 -- and the obvious conversion isn't portable. Going via STR() first works on all three engines; casting the gYear directly works on only one.
What to take away
FILTER constrains; it can't create bindings.
Datatypes are real. xsd:gYear isn't a number, and engines disagree about whether they will convert one for you.
STR() first, then cast. It's the portable idiom, and it's explicit about what's happening.
An expression that fails inside a FILTER removes the row silently. Zero rows is the classic symptom of a datatype mistake, not evidence of an empty dataset.
pattern matches 33 rows
|
v
FILTER( xsd:integer(STR(?founded)) < 1970 )
| -----+-----
| +-- "1979" a plain string
|
+-- true --> row kept
+-- false --> row dropped
|
v
10 rows out
Measured on this dataset, all three engines, same query:
xsd:integer(?founded) editor 0 holos 0 fuseki 10
?founded < "1970"^^xsd:gYear editor 0 holos 0 fuseki 10
xsd:integer(STR(?founded)) editor 10 holos 10 fuseki 10 ok
STR(?founded) < "1970" editor 10 holos 10 fuseki 10 ok
The unportable versions do not error. They return zero rows and
look like a fact about the data.
Editor10HOLOS10Fuseki10bookshop-trail-1.1.ttl
Q08
Large shops with a cafe
Which shops have both a cafe and more than 150 square metres of floor?
# Q08 Large shops with a cafe
# Which shops have both a cafe and more than 150 square metres of floor?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?name ?area
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name ;
bs:hasCafe ?cafe ;
bs:floorArea ?area .
FILTER( ?cafe && ?area > 150 )
}
ORDER BY DESC(?area)
Two conditions combined with &&. Booleans in the data are real xsd:boolean values, so ?cafe can be tested directly without comparing it to anything.
What to take away
A boolean-valued variable can be used as a condition directly.
&& and || combine conditions; ! negates one.
Writing `?cafe = true` works but reads worse, and tells the reader you weren't sure it was a boolean.
?shop --bs:hasCafe---> ?cafe true / false
|
+----bs:floorArea--> ?area a decimal
FILTER( ?cafe && ?area > 150 )
^ ^
| +-- comparison yields a boolean
+-- already a boolean; no "= true" needed
&& short-circuits, and an error on the right of a false && is
swallowed. That is deliberate, and occasionally useful.
Editor13HOLOS13Fuseki13bookshop-trail-1.1.ttl
Q09
Sort books into price bands
Group the books into cheap, mid and dear without changing the data.
BIND computes a value and binds it to a new variable, which can then be selected, sorted or grouped like any other. Nested IF builds the band. Unlike FILTER, BIND adds a column rather than removing rows.
What to take away
BIND adds a computed column; FILTER removes rows. They are the two halves of expression handling.
IF(condition, then, else) nests, and is the closest SPARQL gets to a CASE statement.
Position matters: BIND can only use what's already bound above it.
?book --bs:rrp--> ?price
BIND( IF(?price < 12, "cheap",
IF(?price < 18, "mid", "dear")) AS ?band )
|
v
+--------+-------+--------+
| ?book | ?price| ?band | <- a new column, computed
+--------+-------+--------+
| ... | 9.99 | cheap |
| ... | 15.50 | mid |
| ... | 21.00 | dear |
+--------+-------+--------+
BIND sees only variables bound EARLIER in the group. Move it to
the top and ?price is unbound, so ?band comes out unbound too.
Editor74HOLOS74Fuseki74bookshop-trail-1.1.ttl
Q10
Titles containing a word
Which books have 'sea' or 'water' somewhere in the title?
# Q10 Titles containing a word
# Which books have 'sea' or 'water' somewhere in the title?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?title
WHERE {
?work a bs:Work ;
rdfs:label ?title .
FILTER( REGEX(?title, "sea|water", "i") )
}
ORDER BY ?title
REGEX matches a regular expression against a string. The third argument 'i' makes it case-insensitive. Because REGEX has to look inside every title, it can't use an index -- fine on 74 books, something to think about on 74 million.
What to take away
REGEX(text, pattern, flags) is the general string test; CONTAINS, STRSTARTS and STRENDS are the cheap specific ones.
REGEX works on the lexical form, so a language-tagged literal and a plain one behave the same way here.
Regular expressions defeat text indexes. Prefer the specific functions when they will do.
?work --rdfs:label--> ?title 74 titles
FILTER( REGEX(?title, "sea|water", "i") )
---+----- -+-
| +-- flags: i = ignore case
+-- alternation: either word
"The Dark Sea" ok "High Water" ok
"Cold Harbour" NO "Scree" NO
CONTAINS(?title, "Sea") is cheaper when you do not need a pattern.
Editor3HOLOS3Fuseki3bookshop-trail-1.1.ttl
Q127
Taking a string apart
Build a short sortable code for each shop from its name.
The string functions compose, and this uses most of them at once: STRLEN to measure, UCASE and SUBSTR to cut, STRBEFORE and STRAFTER to split on a marker, CONTAINS and STRSTARTS to test, CONCAT to put the pieces back together.
What to take away
SUBSTR counts from 1 and takes a length, not an end position. afn:substr does neither -- see q103.
CONTAINS and STRSTARTS say what REGEX would, cost less, and read better. Keep REGEX for actual patterns.
STRBEFORE and STRAFTER return the empty string when the marker is absent. Test with CONTAINS first if that matters.
"The Harbour Page"
STRSTARTS(?n, "The ") true
STRAFTER(?n, "The ") "Harbour Page" drop the article
STRBEFORE(?bare, " ") "Harbour" first word
SUBSTR(?first, 1, 3) "Har"
UCASE( that ) "HAR"
STRLEN(?n) 16
STRENDS(?n, "Books") false -> "-"
CONCAT("HAR", "-", "16", "-") "HAR-16-"
+--------------------+------------------+--------+
| The Harbour Page | HAR-16- | 16 |
| The Inkwell | INK-11- | 11 |
| Castle Steps Books | CAS-18B | 18 |
+--------------------+------------------+--------+
the whole family, and what each is for:
STRLEN length in characters, not bytes
SUBSTR by position; 1-based, and takes a LENGTH
UCASE LCASE case, for comparison rather than display
STRSTARTS prefix test STRENDS suffix test
CONTAINS substring test -- cheaper than REGEX (q10)
STRBEFORE everything before the first match; "" if absent
STRAFTER everything after it; "" if absent
CONCAT joins, and propagates the language tag only when
every argument carries the same one
STRBEFORE and STRAFTER returning "" rather than erroring when
the marker is missing is the trap: an empty string is a
perfectly good value, and it will flow through the rest of the
expression without complaint.
Editor10HOLOS10Fuseki10bookshop-trail-1.1.ttl
Q11
Place names in Welsh and Gaelic
Which places carry a name in a language other than English?
# Q11 Place names in Welsh and Gaelic
# Which places carry a name in a language other than English?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?place ?label ?language
WHERE {
?place a bs:Place ;
rdfs:label ?label .
BIND( LANG(?label) AS ?language )
FILTER( ?language != "en" && ?language != "" )
}
ORDER BY ?language ?label
A language-tagged literal carries its tag as part of the value. LANG() extracts it; comparing it to '' finds the untagged ones. Here we keep everything that's tagged but not English.
What to take away
A language tag is part of the literal, not a separate property.
LANG() returns the empty string for a literal with no tag, so a bare != "en" would let untagged strings through.
langMatches handles subtags such as cy-GB properly; use it when the data may contain them.
bt:place-cardiff rdfs:label "Cardiff"@en
rdfs:label "Caerdydd"@cy
------- --
value tag
LANG(?label) --> "en" or "cy" or "" (no tag at all)
FILTER( LANG(?label) != "en" && LANG(?label) != "" )
langMatches(LANG(?l), "cy") is the proper test: it also matches
"cy-GB", which a plain = would miss.
# Q12 Build a one-line description of each shop
# Produce a single human-readable string per shop.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?description
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name ;
bs:founded ?founded ;
bs:locatedIn ?place .
?place rdfs:label ?town .
FILTER( LANG(?town) = "en" )
BIND( CONCAT(?name, " of ", ?town, ", est. ", STR(?founded)) AS ?description )
}
ORDER BY ?description
String functions compose. CONCAT joins, UCASE and SUBSTR reshape, STR strips a literal down to its lexical form so that a typed value can be glued to a plain one without a datatype clash.
What to take away
STR() is the workhorse: it turns any literal or IRI into a plain string so the string functions will accept it.
CONCAT with a typed literal is a common source of silently empty results.
Everything computed with BIND can be selected, ordered and grouped like stored data.
"The Inkwell" + "Wigtown" + 1979
| | |
| | +- STR() -> "1979"
| | (drop the xsd:gYear)
v v v
CONCAT(?name, " of ", ?town, ", est. ", STR(?founded))
|
v
"The Inkwell of Wigtown, est. 1979"
Without STR() around the gYear, CONCAT is given a typed literal
and an engine may refuse the whole expression.
# Q13 How old is each shop today
# How many years has each shop been trading?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
SELECT ?name ?founded ?age
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name ;
bs:founded ?founded .
BIND( YEAR(NOW()) - xsd:integer(STR(?founded)) AS ?age )
}
ORDER BY DESC(?age) ?name
LIMIT 12
NOW() gives the current dateTime, YEAR() pulls the year out of it, and subtracting the founding year gives an age that changes as the calendar does. The result is computed at query time and stored nowhere.
What to take away
NOW() is evaluated once per query, so every row sees the same instant.
The same gYear trap as q07, and here it's worse: a failed cast inside BIND leaves ?age unbound and keeps the row, so the query returns the right number of rows with an empty column.
YEAR, MONTH, DAY, HOURS, MINUTES and SECONDS take apart a dateTime.
Deriving values at query time is usually better than storing them, because stored ages go stale.
NOW() --> 2026-09-09T... --YEAR()--> 2026
|
bs:founded "1979"^^xsd:gYear |
| |
+- xsd:integer(STR()) --> 1979 |
| |
+---- - ------+
|
v
?age = 47
Derived, not stored. Run it next year and every number moves.
Editor12HOLOS12Fuseki12bookshop-trail-1.1.ttl
Q128
Asking a value what it is
For one shop, report the kind and datatype of everything said about it.
# Q128 Asking a value what it is
# For one shop, report the kind and datatype of everything said about it.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
SELECT ?kind ?datatype (COUNT(*) AS ?n)
WHERE {
bt:shop-inkwell ?p ?o .
BIND( IF( isIRI(?o), "IRI",
IF( isBLANK(?o), "blank node",
IF( isNUMERIC(?o), "numeric literal",
"literal" ))) AS ?kind )
BIND( IF( isLITERAL(?o), DATATYPE(?o), "" ) AS ?datatype )
}
GROUP BY ?kind ?datatype
ORDER BY ?kind ?datatype
Four tests sort every RDF term between them -- isIRI, isBLANK, isLITERAL and isNUMERIC -- and DATATYPE names the type of a literal. This is what to run when a FILTER is not matching and you cannot see why, because the answer is almost always that the value is not the type you assumed.
What to take away
isIRI, isBLANK, isLITERAL and isNUMERIC partition every RDF term. DATATYPE names a literal's type.
A language-tagged literal is rdf:langString, not xsd:string. Most surprising empty results come from this.
langMatches handles language ranges; equality on LANG() does not. STRDT and STRLANG build typed and tagged literals.
for each object of bt:shop-inkwell:
isIRI(?o) -> IRI the town it is in
isLITERAL(?o) -> literal its name, year, size
isNUMERIC(?o) -> numeric a subset of the literals
isBLANK(?o) -> blank node none here; see module 16
DATATYPE(?o) the type IRI of a literal.
everything this dataset says about The Inkwell, sorted:
+-----------------+------------------+----+
| IRI | | 11 |
| literal | rdf:langString | 1 | the name
| literal | xsd:anyURI | 1 | the website
| literal | xsd:boolean | 2 | cafe, secondhand
| literal | xsd:gYear | 1 | founded
| numeric literal | xsd:decimal | 3 | coordinates
| numeric literal | xsd:integer | 1 | floor area
+-----------------+------------------+----+
Note what is NOT there: a single xsd:string. Every string in
this dataset carries either a language tag or a datatype, and
that is deliberate.
+----------------------------------------------------------+
| A language-tagged literal has datatype rdf:langString, |
| never xsd:string. A FILTER written as |
| FILTER( DATATYPE(?label) = xsd:string ) |
| therefore matches none of the labels in this dataset, |
| and that is q11's lesson from the other direction. |
+----------------------------------------------------------+
langMatches is the right test for a language tag, because it
understands ranges: langMatches(LANG(?x), "en") also matches
"en-GB", where LANG(?x) = "en" does not.
STRDT and STRLANG go the other way -- they build a typed or
tagged literal from a plain string, which is how you repair
data that arrived untyped.
Editor7HOLOS7Fuseki7bookshop-trail-1.1.ttl
Q129
Four ways to lose the decimals
Round the shelf price of every book, four different ways, and see where they differ.
# Q129 Four ways to lose the decimals
# Round the shelf price of every book, four different ways, and see where they differ.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
SELECT ?rounded ?floored ?ceiled ?absolute ?randInRange (COUNT(*) AS ?books)
WHERE {
?record bs:shelfPrice ?price .
BIND( ROUND(?price) AS ?rounded )
BIND( FLOOR(?price) AS ?floored )
BIND( CEIL(?price) AS ?ceiled )
BIND( ABS(0 - ?price) AS ?absolute )
BIND( IF( RAND() >= 0.0 && RAND() < 1.0, "yes", "impossible" )
AS ?randInRange )
}
GROUP BY ?rounded ?floored ?ceiled ?absolute ?randInRange
ORDER BY ?absolute
LIMIT 8
ROUND, FLOOR, CEIL and ABS are the whole of SPARQL's numeric toolkit, and the differences between the first three only show up on particular values -- which is why they are worth putting side by side once rather than reading about.
What to take away
ROUND, FLOOR, CEIL and ABS are the numeric functions. There is no sqrt, no pow and no log in standard SPARQL.
Rounding negative numbers is where engines have historically differed. The specification says ties go toward positive infinity.
RAND() is a real random double in [0, 1). Anything using it is unrepeatable, so keep it out of anything you need to verify.
value ROUND FLOOR CEIL ABS
----- ----- ----- ---- ---
8.99 9 8 9 8.99
9.50 10 9 10 9.50
12.00 12 12 12 12.00
-3.50 -3 -4 -3 3.50 <- the interesting row
ROUND takes half away from zero on some engines and to
positive infinity on others; SPARQL says "to nearest, ties
toward positive infinity", so -3.5 rounds to -3. Test it
rather than assuming, if the sign matters.
+----------------------------------------------------------+
| There is no square root and no power. Module 09 is built |
| around that fact, and module 15 shows what afn:sqrt and |
| math:pow cost you in portability. |
+----------------------------------------------------------+
Note the type. ROUND of an xsd:decimal is an xsd:decimal, so
the answers come back as 8.0 and 12.0 rather than 8 and 12.
Use xsd:integer() if you want the whole number.
RAND() returns a double in [0, 1). It is genuinely random, so
a query using it cannot be checked by comparing answers --
which is why this one asks whether the value is in range
rather than what it is.
# Q130 Pulling a date apart
# Break every event date into its parts.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
SELECT ?year ?month (COUNT(*) AS ?events)
WHERE {
?event a bs:Event ;
bs:eventDate ?date .
# xsd:date, promoted to xsd:dateTime so that every engine can read
# it -- see the diagram. STRDT builds a typed literal from a string.
BIND( STRDT( CONCAT( STR(?date), "T00:00:00" ), xsd:dateTime ) AS ?at )
BIND( YEAR(?at) AS ?year )
BIND( MONTH(?at) AS ?month )
BIND( DAY(?at) AS ?day )
}
GROUP BY ?year ?month
ORDER BY ?year ?month
PREFIXbs:<https://example.org/bookshop-trail/schema#>PREFIXxsd:<http://www.w3.org/2001/XMLSchema#>SELECT?year?month (COUNT(*) AS?events)
WHERE {
?eventabs:Event ;
bs:eventDate?date .
# xsd:date, promoted to xsd:dateTime so that every engine can read# it -- see the diagram. STRDT builds a typed literal from a string.BIND( STRDT( CONCAT( STR(?date), "T00:00:00" ), xsd:dateTime ) AS?at )
BIND( YEAR(?at) AS?year )
BIND( MONTH(?at) AS?month )
BIND( DAY(?at) AS?day )
}
GROUPBY?year?monthORDERBY?year?month
How it works
A xsd:dateTime or xsd:date can be taken apart with YEAR, MONTH, DAY and, where there is a time, HOURS, MINUTES, SECONDS and TIMEZONE. They return plain numbers, which is what makes grouping by month possible at all -- there is no date arithmetic in SPARQL beyond this.
What to take away
YEAR, MONTH, DAY, HOURS, MINUTES, SECONDS and TIMEZONE take a date apart into numbers -- reliably, on all three engines, only for xsd:dateTime. STRDT is how you widen an xsd:date to one.
There is no date arithmetic and no formatting. Grouping and comparison are what the parts are for.
NOW() is fixed for the whole query, so two calls in one query always agree -- which is what makes it usable at all.
"2024-06-15"^^xsd:date
YEAR -> 2024 MONTH -> 6 DAY -> 15
and for a dateTime, three more:
HOURS MINUTES SECONDS TIMEZONE / TZ
what you can do with them:
GROUP BY (MONTH(?d)) events per month
FILTER( YEAR(?d) = 2024 ) a year's worth
?d < NOW() in the past
what you cannot do:
?d + "P1M"^^xsd:duration no duration arithmetic
DATEDIFF(?a, ?b) no such function
and one thing that is not portable, measured:
YEAR( "2025-03-08"^^xsd:date ) editor 2025 fuseki 2025
holos UNBOUND
YEAR( "...T00:00:00"^^xsd:dateTime ) all three 2025
HOLOS implements the date functions for xsd:dateTime only, and
returns the row with the variable unbound rather than raising --
the failure mode module 15 keeps warning about. So this query
promotes the date first:
STRDT( CONCAT( STR(?date), "T00:00:00" ), xsd:dateTime )
which is the portable way to widen a date, and incidentally the
clearest use of STRDT there is.
Subtracting two xsd:dateTime values gives an xsd:duration on
engines that support it and an error on those that do not, so
q13 computes an age from YEAR() alone. That is the portable
way, and it is off by up to a year -- which the query says.
+----------------------------------------------------------+
| MONTH returns 6, not "June" and not "06". Formatting a |
| date for display is not something SPARQL does; do it in |
| whatever consumes the results. |
+----------------------------------------------------------+
Editor11HOLOS11Fuseki11bookshop-trail-1.1.ttl
Module 03
Optional data, alternatives and negation
Real data has holes. OPTIONAL keeps a row when the extra fact is missing, UNION merges two shapes, and MINUS and NOT EXISTS remove rows -- in ways that aren't quite interchangeable. VALUES is the other half of the same idea: rather than filtering rows out, it supplies the ones you want.
# Q14 Shops, with a website if there's one
# List every shop, showing its website where one is recorded.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?name ?site
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name .
OPTIONAL { ?shop bs:website ?site . }
}
ORDER BY ?name
Without OPTIONAL, the pattern would silently drop the shops that have no website. OPTIONAL tries the inner pattern and, when it does not match, keeps the row anyway with ?site left unbound.
What to take away
OPTIONAL is a left join. The left side is kept whatever happens on the right.
An unbound variable isn't an empty string and not zero. It's absent, and BOUND() is how you test for it.
The commonest bug in SPARQL is a missing OPTIONAL quietly shrinking the answer.
required optional
+--------------------+ +----------------------+
| ?shop a bs:Bookshop|----->| ?shop bs:website ?site|
| ?shop rdfs:label ? | +----------------------+
+--------------------+ |
33 rows +---------+----------+
v v
matched: ?site bound no match:
?site UNBOUND
row still kept
|
v
33 rows
Drop the OPTIONAL and you get fewer rows -- and no warning.
Editor33HOLOS33Fuseki33bookshop-trail-1.1.ttl
Q15
Which books have no ISBN
Find the books with no ISBN recorded, and say why.
# Q15 Which books have no ISBN
# Find the books with no ISBN recorded, and say why.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?title ?year
WHERE {
?work a bs:Work ;
rdfs:label ?title ;
bs:publicationYear ?year .
OPTIONAL { ?work bs:isbn ?isbn . }
FILTER( !BOUND(?isbn) )
}
ORDER BY ?year
OPTIONAL brings the ISBN in when there's one; !BOUND(?isbn) then keeps only the rows where it did not arrive. This 'optional then test for unbound' shape is the classic way to express absence, and still the clearest when you also want a value from the row.
What to take away
OPTIONAL + !BOUND is the 'anti-join': rows where the optional part failed.
NOT EXISTS says the same thing more directly when you don't need anything from inside the optional block.
Absence in the data usually means something. Here it means the book is older than the ISBN system.
?work rdfs:label ?title 74 works
|
v
OPTIONAL { ?work bs:isbn ?isbn }
|
+-------+--------+
v v
?isbn bound ?isbn UNBOUND
| |
| v
| FILTER(!BOUND(?isbn)) ok kept
v
dropped
Every book published before 1970 lands on the right-hand branch,
because ISBNs did not exist yet.
Editor11HOLOS11Fuseki11bookshop-trail-1.1.ttl
Q16
Towns with no bookshop
Which settlements on the map have no bookshop at all?
# Q16 Towns with no bookshop
# Which settlements on the map have no bookshop at all?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?townName
WHERE {
?town a bs:Settlement ;
rdfs:label ?townName .
FILTER( LANG(?townName) = "en" )
FILTER NOT EXISTS { ?shop bs:locatedIn ?town . }
}
ORDER BY ?townName
NOT EXISTS tests whether a pattern has any match, using the current row's bindings. It binds nothing itself -- it's a pure test -- so ?shop inside the braces is invisible outside them.
What to take away
NOT EXISTS asks 'is there any match?' and contributes no bindings.
It's correlated: variables bound outside are visible inside.
Prefer it to OPTIONAL + !BOUND when you want nothing from the inner pattern -- it says what you mean.
for each ?town:
+------------------------------------------+
| does ANY ?shop have bs:locatedIn ?town? |
+------------------------------------------+
| |
yes no
| |
dropped ok kept
Durham, Perth, Fort William, Truro
NOT EXISTS is evaluated per row, with ?town already bound. That
is what makes it a correlated test rather than a set difference.
Editor4HOLOS4Fuseki4bookshop-trail-1.1.ttl
Q17
MINUS and NOT EXISTS aren't the same
Show the case where swapping MINUS for NOT EXISTS changes the answer.
# Q17 MINUS and NOT EXISTS aren't the same
# Show the case where swapping MINUS for NOT EXISTS changes the answer.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
SELECT ?viaNotExists ?viaMinusShared ?viaMinusUnrelated
WHERE {
# Shops with no cafe, asked three ways. Only two of them work.
{
SELECT (COUNT(*) AS ?viaNotExists) WHERE {
?s a bs:Bookshop .
FILTER NOT EXISTS { ?s bs:hasCafe true . }
}
}
{
SELECT (COUNT(*) AS ?viaMinusShared) WHERE {
?s a bs:Bookshop .
MINUS { ?s bs:hasCafe true . }
}
}
{
SELECT (COUNT(*) AS ?viaMinusUnrelated) WHERE {
?s a bs:Bookshop .
MINUS { ?other bs:hasCafe true . }
}
}
}
PREFIXbs:<https://example.org/bookshop-trail/schema#>SELECT?viaNotExists?viaMinusShared?viaMinusUnrelatedWHERE {
# Shops with no cafe, asked three ways. Only two of them work.
{
SELECT (COUNT(*) AS?viaNotExists) WHERE {
?sabs:Bookshop .
FILTERNOTEXISTS { ?sbs:hasCafe true . }
}
}
{
SELECT (COUNT(*) AS?viaMinusShared) WHERE {
?sabs:Bookshop .
MINUS { ?sbs:hasCafe true . }
}
}
{
SELECT (COUNT(*) AS?viaMinusUnrelated) WHERE {
?sabs:Bookshop .
MINUS { ?otherbs:hasCafe true . }
}
}
}
How it works
NOT EXISTS evaluates its pattern with the outer bindings in place. MINUS removes rows by comparing whole solutions, and a MINUS whose pattern shares no variable with the outer query can remove nothing at all. This query runs both side by side so the difference is visible rather than theoretical.
What to take away
NOT EXISTS is a test on the current row. MINUS is a set operation on whole solutions.
MINUS with no shared variable is a silent no-op -- it won't error, it will just fail to filter.
When in doubt use NOT EXISTS: its correlation behaviour is the one people expect.
NOT EXISTS { ?shop bs:hasCafe true }
?shop is BOUND inside -> a real per-row test
--> removes the shops that do have a cafe
MINUS { ?other bs:hasCafe true }
no shared variable -> nothing to compare on
--> removes NOTHING
+--------------+-----------+------------+
| | shares a | removes |
| | variable? | |
+--------------+-----------+------------+
| NOT EXISTS | n/a | correctly |
| MINUS (same) | yes | correctly |
| MINUS (diff) | no | nothing |
+--------------+-----------+------------+
Editor1HOLOS1Fuseki1bookshop-trail-1.1.ttl
Q18
Everyone who worked on a book
List every person credited on a work, whether as author or translator.
# Q18 Everyone who worked on a book
# List every person credited on a work, whether as author or translator.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT DISTINCT ?personName ?role
WHERE {
{
?work bs:author ?person .
BIND( "author" AS ?role )
}
UNION
{
?work bs:translatedBy ?person .
BIND( "translator" AS ?role )
}
?person rdfs:label ?personName .
}
ORDER BY ?personName ?role
UNION evaluates both branches independently and concatenates the results. The two branches need not bind the same variables; here they deliberately bind ?role differently so the output says which branch a row came from.
What to take away
UNION concatenates; it doesn't merge or deduplicate.
Branches may bind different variables. Anything a branch doesn't bind comes out unbound for its rows.
Binding a constant per branch is the standard trick for labelling which alternative matched.
+--------------------------+
| ?work bs:author ?person | --> ?role = "author"
| |
+--------------------------+
UNION both result sets
+--------------------------+ concatenated
| ?work bs:translatedBy ?p | --> ?role = "translator"
+--------------------------+
UNION does NOT deduplicate. Add DISTINCT if you need that.
A person appearing in both branches gets two rows, which here
is exactly right: they did two different jobs.
Editor36HOLOS36Fuseki36bookshop-trail-1.1.ttl
Q19
Why a FILTER inside OPTIONAL behaves oddly
Compare filtering inside an OPTIONAL with filtering after it.
A FILTER inside OPTIONAL decides whether the optional part matches. A FILTER after it decides whether the whole row survives. The first keeps every shop and blanks out the small ones; the second throws the small ones away entirely.
What to take away
A FILTER inside OPTIONAL constrains the optional match. Outside, it constrains the row.
A comparison on an unbound variable is an error, and an error in a FILTER means 'drop the row'.
If your OPTIONAL suddenly stops being optional, look for a FILTER that escaped from its braces.
INSIDE AFTER
OPTIONAL { OPTIONAL {
?shop bs:floorArea ?a ?shop bs:floorArea ?a
FILTER(?a > 250) }
} FILTER(?a > 250)
33 rows out 8 rows out
small shops kept, small shops REMOVED --
?a unbound because an unbound ?a
fails the comparison
Same words, different place, different answer. This one catches
everybody at least once.
VALUES puts a small table of bindings directly in the query. The engine treats it as data, joins it to everything else, and -- this is the part that matters -- it restricts the pattern before the join rather than filtering rows afterwards.
What to take away
VALUES supplies an inline table of bindings and joins it like any other pattern.
Prefer it to FILTER(?x IN ...) when you know the values up front: it narrows the search rather than filtering the results.
It is the natural way to parameterise a query from code, and the mechanism SERVICE uses to pass bindings to a remote endpoint.
VALUES ?shop {
bt:shop-inkwell
bt:shop-ex-libris
bt:shop-sea-margin
}
?shop rdfs:label ?name ; bs:founded ?founded .
the VALUES block IS a result table, written by hand:
+--------------------+
| ?shop |
+--------------------+
| bt:shop-inkwell | 3 rows in,
| bt:shop-ex-libris | joined to the pattern
| bt:shop-sea-margin |
+--------------------+
three ways to say "one of these", and they are not equal:
VALUES ?shop { ... } 3 shops looked up
FILTER( ?shop IN (a, b, c) ) 33 matched, then 30 discarded
{ } UNION { } UNION { } three separate patterns
VALUES restricts BEFORE the join. On 33 shops nobody notices; on
a few million the difference is the query finishing or not.
It is also how you parameterise: generate the VALUES block from
your program and the rest of the query never changes.
Editor3HOLOS3Fuseki3bookshop-trail-1.1.ttl
Q131
IN, NOT IN, and when to use VALUES instead
Find the shops in three named towns, then everything outside them.
# Q131 IN, NOT IN, and when to use VALUES instead
# Find the shops in three named towns, then everything outside them.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
SELECT ?where (COUNT(*) AS ?shops)
WHERE {
?shop a bs:Bookshop ;
bs:locatedIn ?town .
BIND( IF( ?town IN ( bt:place-hay-on-wye,
bt:place-wigtown,
bt:place-sedbergh ),
"a book town", "somewhere else" ) AS ?where )
FILTER( ?town NOT IN ( bt:place-york ) )
}
GROUP BY ?where
ORDER BY ?where
IN tests whether a value is one of a list, and NOT IN is its negation. Both are expressions, so they live in a FILTER and can only narrow what a pattern already found. VALUES is a pattern, so it can supply rows rather than filter them -- which is the difference that decides which to reach for.
What to take away
IN and NOT IN are expressions and belong in a FILTER. They narrow; they cannot supply rows.
VALUES does the positive case better, because the engine can start from the list rather than filter down to it. Compare q98.
NOT IN has no VALUES equivalent. For a negative, that is what MINUS and NOT EXISTS are for.
FILTER( ?town IN (bt:place-hay-on-wye,
bt:place-wigtown,
bt:place-sedbergh) )
an EXPRESSION. The pattern has already found every shop;
this throws most of them away.
VALUES ?town { bt:place-hay-on-wye
bt:place-wigtown
bt:place-sedbergh }
a PATTERN. Three rows exist before the join runs, and the
engine can start from them.
+--------------+---------------------------+------------------+
| | reads as | engine can |
+--------------+---------------------------+------------------+
| IN | filter, after the match | rarely optimise |
| VALUES | data, before the match | drive the join |
| NOT IN | filter, after the match | no equivalent |
+--------------+---------------------------+------------------+
NOT IN has no VALUES form, because a negative has no rows to
supply. MINUS and NOT EXISTS are the pattern-shaped negations
(q17), and IN is a list of values rather than a graph pattern.
Two more things worth knowing. IN evaluates its list left to
right and stops at the first match, so put the likely ones
first if the list is long. And an error in the list -- an
unbound variable, say -- makes the whole expression an error
rather than false, which is q19's territory.
Editor2HOLOS2Fuseki2bookshop-trail-1.1.ttl
Module 04
Counting, grouping and summarising
GROUP BY collapses many rows into one per group. HAVING filters the groups. The trap that catches everyone is putting an aggregate in the wrong place, and this module walks straight into it on purpose.
# Q20 How many shops in each town
# Count the bookshops town by town.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?townName (COUNT(?shop) AS ?shops)
WHERE {
?shop a bs:Bookshop ;
bs:locatedIn ?town .
?town rdfs:label ?townName .
FILTER( LANG(?townName) = "en" )
}
GROUP BY ?townName
ORDER BY DESC(?shops) ?townName
GROUP BY collapses the matching rows into one row per distinct ?townName, and COUNT reports how many went into each. Every expression in SELECT must then be either a grouping key or an aggregate -- there's nowhere else for a value to come from.
What to take away
GROUP BY turns many rows into one row per key.
Anything in SELECT must be a grouping key or wrapped in an aggregate.
COUNT(?x) counts rows where ?x is bound; COUNT(*) counts rows.
rows after matching after GROUP BY ?townName
+----------+----------+ +----------+-------+
| Wigtown | Inkwell | | Wigtown | 2 |
| Wigtown |Marginalia| --> | Edinburgh| 2 |
| Edinburgh| Colophon | | Hay-on-W | 2 |
| Edinburgh| Broken S | | York | 2 |
| York | Endpapers| | ... | ... |
| ... | ... | +----------+-------+
+----------+----------+ one row per group
SELECT may name ?townName (the key) and COUNT(...) (an aggregate).
Naming ?shop would be a syntax error: which of the two?
# Q21 What each shop's stock is worth
# Total the shelf value of every shop's stock.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?name (ROUND(SUM(?copies * ?price) * 100) / 100 AS ?stockValue)
WHERE {
?record a bs:StockRecord ;
bs:atShop ?shop ;
bs:copies ?copies ;
bs:shelfPrice ?price .
?shop rdfs:label ?name .
}
GROUP BY ?shop ?name
ORDER BY DESC(?stockValue)
LIMIT 12
The value of a stock line is copies times price, so the aggregate has to multiply before it sums. SUM accepts an expression, not just a variable, which saves a BIND -- though a BIND would be clearer if the expression got any longer.
What to take away
Aggregates take expressions, so arithmetic can happen before the sum.
ROUND, FLOOR, CEIL and ABS are available for tidying the result.
Grouping by ?shop and selecting ?name works because each shop has exactly one label -- if it had two, you would need SAMPLE or a second grouping key.
each bs:StockRecord: copies x shelfPrice
12 x 9.99 = 119.88
8 x 10.99 = 87.92
5 x 8.99 = 44.95
---------
SUM per shop, after GROUP BY ?shop 252.75
SELECT ?name (SUM(?copies * ?price) AS ?value)
---------+-------
expression, evaluated per row,
THEN summed per group
# Q22 Average attendance by kind of event
# Which kinds of event draw the biggest crowds?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
SELECT ?kind
(COUNT(?event) AS ?events)
(ROUND(AVG(?attendance)) AS ?meanAttendance)
(MIN(?attendance) AS ?smallest)
(MAX(?attendance) AS ?largest)
WHERE {
?event a bs:Event ;
bs:eventKind ?kind ;
bs:attendance ?attendance .
}
GROUP BY ?kind
ORDER BY DESC(?meanAttendance)
One row per event goes in; one row per event kind comes out, carrying the mean, the extremes and the count. Reporting the count alongside the average is a habit worth forming: an average over two events means much less than one over twenty.
What to take away
COUNT, SUM, AVG, MIN, MAX and SAMPLE can all appear in one SELECT over the same grouping.
Always show the group size next to an average.
MIN and MAX work on any ordered type, including dates and strings.
59 events
|
+- Launch --+
+- Reading --+ GROUP BY ?kind
+- Panel --+ |
+- Workshop --+ v
+- ... --+ +---------+-----+-----+-----+-----+
| kind | n | avg | min | max |
+---------+-----+-----+-----+-----+
| Launch | 8 | 190 | 102 | 320 |
| Workshop| 8 | 21 | 14 | 28 |
+---------+-----+-----+-----+-----+
Several aggregates over the same grouping cost one pass.
Editor7HOLOS7Fuseki7bookshop-trail-1.1.ttl
Q23
Only the busy shops
Which shops held three or more events, and how many people came in total?
# Q23 Only the busy shops
# Which shops held three or more events, and how many people came in total?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?name (COUNT(?event) AS ?events) (SUM(?attendance) AS ?totalAudience)
WHERE {
?event a bs:Event ;
bs:heldAt ?shop ;
bs:attendance ?attendance .
?shop rdfs:label ?name .
}
GROUP BY ?shop ?name
HAVING( COUNT(?event) >= 3 )
ORDER BY DESC(?totalAudience)
HAVING filters groups, after aggregation. FILTER can't do this job: it runs before the grouping, when the aggregate doesn't exist yet. The two aren't interchangeable and the error message when you confuse them is rarely helpful.
What to take away
FILTER runs before GROUP BY; HAVING runs after it.
An aggregate in a FILTER is an error. Aggregates don't exist until the grouping has happened.
HAVING may use an aggregate you did not select.
WHERE -> filter individual rows (FILTER lives here)
|
v
GROUP BY -> collapse into groups
|
v
HAVING -> filter whole groups (HAVING lives here)
|
v
ORDER BY -> sort what survived
HAVING( COUNT(?event) >= 3 )
--------+------
an aggregate -- only legal after grouping
Editor6HOLOS6Fuseki6bookshop-trail-1.1.ttl
Q24
List each author's books on one line
For each author, put all their titles into a single cell.
# Q24 List each author's books on one line
# For each author, put all their titles into a single cell.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?author (COUNT(?book) AS ?titles)
(GROUP_CONCAT(?title; SEPARATOR=" / ") AS ?works)
WHERE {
?book a bs:Work ;
rdfs:label ?title ;
bs:author ?person .
?person rdfs:label ?author .
}
GROUP BY ?person ?author
ORDER BY DESC(?titles) ?author
LIMIT 12
GROUP_CONCAT folds the values of a group into one string with a separator. It's the aggregate you reach for when the consumer of the result wants a summary rather than a row per item.
What to take away
GROUP_CONCAT(?x; SEPARATOR=", ") flattens a group into one string.
The order inside the concatenation isn't guaranteed unless you sort in a sub-query first.
COUNT alongside it tells the reader how many items were folded in.
Agnes Varden -+- "Minster Yard"
+- "The Chapter House"
+- "A Cold Coming"
|
GROUP_CONCAT(?title; SEPARATOR=" / ")
|
v
"Minster Yard / The Chapter House / A Cold Coming"
The separator is a keyword argument after a semicolon -- the one
place in SPARQL where that syntax appears.
Engines differ, and that's expected. the ORDER of the titles inside the concatenated string isn't specified, and the three engines really do differ. The set of titles is identical; only the sequence varies. If order matters, sort in a sub-query first -- and even then, not every engine promises to honour it.
Editor12HOLOS12Fuseki12bookshop-trail-1.1.ttl
Q25
Counting things that aren't there
Count events per shop, including the shops that held none.
# Q25 Counting things that aren't there
# Count events per shop, including the shops that held none.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?name (COUNT(?event) AS ?events)
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name .
OPTIONAL { ?event bs:heldAt ?shop . }
}
GROUP BY ?shop ?name
ORDER BY ?events ?name
A plain GROUP BY over the events can only see shops that appear in an event, so a shop with none is simply absent. Bringing the shops in first and making the events OPTIONAL keeps every shop, and COUNT(?event) then correctly reports zero -- because COUNT ignores unbound values.
What to take away
Aggregating over a join can only see what the join produced. Zeroes have to be arranged for.
COUNT(?x) ignores unbound ?x; COUNT(*) counts the row regardless. Here that distinction is the whole answer.
In this dataset every shop has held at least one event, so the two forms agree -- run it against a shop you've just added and they won't.
WRONG RIGHT
?e bs:heldAt ?shop ?shop a bs:Bookshop
GROUP BY ?shop OPTIONAL { ?e bs:heldAt ?shop }
GROUP BY ?shop
shops with 0 events every shop appears
vanish entirely |
v
COUNT(?event) = 0
because COUNT skips unbound
COUNT(?event) counts bound values -> 0 for an empty shop
COUNT(*) counts rows -> 1 ... which is wrong
Editor33HOLOS33Fuseki33bookshop-trail-1.1.ttl
Q26
One number for the whole dataset
How many shops, books, authors and events are there altogether?
# Q26 One number for the whole dataset
# How many shops, books, authors and events are there altogether?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
SELECT ?class (COUNT(DISTINCT ?s) AS ?count)
WHERE {
{ ?s a bs:Bookshop . BIND( "bookshops" AS ?class ) }
UNION
{ ?s a bs:Work . BIND( "works" AS ?class ) }
UNION
{ ?s a bs:Author . BIND( "authors" AS ?class ) }
UNION
{ ?s a bs:Event . BIND( "events" AS ?class ) }
}
GROUP BY ?class
ORDER BY ?class
PREFIXbs:<https://example.org/bookshop-trail/schema#>SELECT?class (COUNT(DISTINCT?s) AS?count)
WHERE {
{ ?sabs:Bookshop . BIND( "bookshops"AS?class ) }
UNION
{ ?sabs:Work . BIND( "works"AS?class ) }
UNION
{ ?sabs:Author . BIND( "authors"AS?class ) }
UNION
{ ?sabs:Event . BIND( "events"AS?class ) }
}
GROUPBY?classORDERBY?class
How it works
An aggregate with no GROUP BY treats the entire result as a single group, giving exactly one row. Four UNION branches each count one class, so the answer arrives as a small summary table rather than as four separate queries.
What to take away
An aggregate with no GROUP BY produces exactly one row.
UNION plus a bound constant is the idiom for a summary table.
COUNT(DISTINCT ?x) is what you want whenever the pattern could match the same resource twice.
+----------------------------+
| ?s a bs:Bookshop -> "shop"|--+
+----------------------------+ |
| ?s a bs:Work -> "work"|--+ UNION
+----------------------------+ +------> GROUP BY ?class
| ?s a bs:Author -> "author"|--+ |
+----------------------------+ | v
| ?s a bs:Event -> "event"|--+ +--------+-----+
+----------------------------+ | author | 32 |
| event | 59 |
| shop | 33 |
| work | 74 |
+--------+-----+
Editor4HOLOS4Fuseki4bookshop-trail-1.1.ttl
Q132
DISTINCT, REDUCED, and what each costs
List the towns that have a bookshop, without repeating any.
# Q132 DISTINCT, REDUCED, and what each costs
# List the towns that have a bookshop, without repeating any.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT REDUCED ?town
WHERE {
?shop a bs:Bookshop ;
bs:locatedIn ?place .
?place rdfs:label ?town .
FILTER( LANG(?town) = "en" )
}
ORDER BY ?town
DISTINCT removes every duplicate, which means the engine has to hold or sort the whole answer to know what it has already seen. REDUCED permits it to remove some and is not obliged to remove any, which lets it drop the adjacent ones as they stream past and keep nothing in memory.
What to take away
DISTINCT guarantees no duplicates and costs memory or a sort. REDUCED permits duplicate removal without requiring it.
A REDUCED result is not reproducible across engines by definition. Never depend on its row count.
COUNT(DISTINCT ?x) is unrelated to the solution modifier and is what you want when counting distinct values.
33 shops -> 26 towns, with duplicates in between
DISTINCT every duplicate gone. Guaranteed.
REDUCED duplicates MAY be gone. Permitted, not required.
+-------------+-------------------+-------------------------+
| | result | cost |
+-------------+-------------------+-------------------------+
| (neither) | 33 rows | none |
| DISTINCT | 26 rows | remembers what it saw |
| REDUCED | 26 to 33 rows | may remember nothing |
+-------------+-------------------+-------------------------+
and this exact query, measured:
editor 26 rows duplicates dropped
holos 26 rows duplicates dropped
fuseki 33 rows duplicates kept
All three are correct. REDUCED permits removal and requires
nothing, so an engine that does the cheap thing and an engine
that does the thorough thing both conform -- and here Jena is
the one that declines to pay. There is no bug to report.
That is what makes REDUCED awkward to teach and awkward to
use: the one guarantee it gives you is that it gives you no
guarantee.
So REDUCED is not "DISTINCT but faster". It is "I do not mind
duplicates, and I would rather you did not pay to remove
them". Use it when the consumer deduplicates anyway, and use
DISTINCT whenever the count matters.
+----------------------------------------------------------+
| COUNT(DISTINCT ?x) is a different thing again, and it |
| does what it says -- q25 uses it. There is no |
| COUNT(REDUCED ?x). |
+----------------------------------------------------------+
Engines differ, and that's expected. and by design. Jena returns all 33 rows; Comunica and HOLOS return the 26 distinct ones. REDUCED permits duplicate removal without requiring it, so every one of those answers conforms.
Editor26HOLOS26Fuseki33bookshop-trail-1.1.ttl
Module 05
Property paths
The feature that turns SPARQL from a table language into a graph language. A path expression walks an arbitrary number of hops, in either direction, and it terminates even when the data has cycles.
# Q27 Every area a shop sits inside
# For one shop, list every containing place all the way up to Great Britain.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?areaName
WHERE {
bt:shop-endpapers bs:locatedIn/bs:within+ ?area .
?area rdfs:label ?areaName .
FILTER( LANG(?areaName) = "en" )
}
bs:within+ follows the containment link one or more times and returns every place it can reach. Writing the same thing as a chain of patterns would need one line per level -- and you would have to know in advance how many levels there are.
What to take away
path+ means one or more hops, and returns every node reachable that way.
A path expression replaces a chain of patterns whose length you do not know.
The result is a set of endpoints, not a route: SPARQL won't tell you which way it went. q32 shows what to do when you need that.
bt:shop-endpapers
| bs:locatedIn
v
place-york --within--> north-yorkshire --within--> yorkshire
| within
v
place-gb <--within-- england
bs:within+ collects EVERY place on that road:
north-yorkshire, yorkshire, england, gb (4 rows)
The + means "one or more hops". The engine keeps walking until
it runs out of edges, and it remembers where it has been.
Editor4HOLOS4Fuseki4bookshop-trail-1.1.ttl
Q28
Why a fixed-length chain gets the wrong answer
Count the shops in each country, first with a fixed chain of hops and then with a path.
# Q28 Why a fixed-length chain gets the wrong answer
# Count the shops in each country, first with a fixed chain of hops and then with a path.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?countryName ?viaFixedChain ?viaPath
WHERE {
?country a bs:Country ; rdfs:label ?countryName .
FILTER( LANG(?countryName) = "en" )
{
SELECT ?country (COUNT(DISTINCT ?shop) AS ?viaFixedChain)
WHERE {
?shop a bs:Bookshop ; bs:locatedIn ?town .
?town bs:within/bs:within/bs:within ?country .
?country a bs:Country .
}
GROUP BY ?country
}
UNION
{
SELECT ?country (COUNT(DISTINCT ?shop) AS ?viaPath)
WHERE {
?shop a bs:Bookshop ; bs:locatedIn ?town .
?town bs:within+ ?country .
?country a bs:Country .
}
GROUP BY ?country
}
}
ORDER BY ?countryName
The place hierarchy is deliberately uneven. An English town sits inside a council area inside a region inside a country; a Scottish or Welsh one sits inside a council area inside the country, with no region in between. A pattern hard-coded to three hops finds England and misses Scotland and Wales entirely -- and reports no error while doing it.
What to take away
Real hierarchies are rarely of uniform depth, and a fixed-length chain silently drops the branches that don't match.
A wrong answer that looks reasonable is worse than an error.
When you mean 'contained in, at any depth', say so with +.
ENGLAND (4 levels) SCOTLAND / WALES (3 levels)
york edinburgh
| within | within
north-yorkshire edinburgh-city
| within | within
yorkshire scotland
| within | within
england gb
| within
gb
?town bs:within/bs:within/bs:within ?country
---- exactly 3 hops ----
matches York -> england ok
misses Edinburgh -> scotland NO (only 2 hops away)
?town bs:within+ ?country matches both
The wrong query returns a plausible, confident, incomplete answer.
That is what makes it dangerous.
Editor4HOLOS4Fuseki4bookshop-trail-1.1.ttl
Q29
Star and plus aren't the same
What's the difference between bs:within* and bs:within+?
# Q29 Star and plus aren't the same
# What's the difference between bs:within* and bs:within+?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?form ?areaName
WHERE {
{
bt:place-york bs:within+ ?area .
BIND( "within+ (one or more)" AS ?form )
}
UNION
{
bt:place-york bs:within* ?area .
BIND( "within* (zero or more)" AS ?form )
}
?area rdfs:label ?areaName .
FILTER( LANG(?areaName) = "en" )
}
ORDER BY ?form ?areaName
* allows zero hops, so the starting node is included in its own answer. + requires at least one hop, so it isn't. The distinction matters most when you're collecting a subtree and need to decide whether the root belongs in it.
What to take away
* is zero-or-more and always includes the starting node.
+ is one-or-more and never includes it, unless a cycle leads back.
? is zero-or-one: the hop is allowed but not required.
starting from place-york:
bs:within+ bs:within*
---------- ----------
york <- zero hops: itself
north-yorkshire north-yorkshire
yorkshire yorkshire
england england
gb gb
4 rows 5 rows
Rule of thumb:
"all my ancestors" -> +
"me and all my ancestors" -> *
The caret reverses the direction of a link, so ^bs:locatedIn goes from a town to the shops in it. Combined with a forward path it gives the whole answer in one expression: down from Wales to its towns, then back up the locatedIn edge to the shops.
What to take away
^p traverses p backwards. It isn't a different property, just a different direction of travel.
A path can mix forward and reverse steps freely.
There's usually a forwards and a backwards way to write the same question; pick the one that reads like the question.
the data points this way:
shop --bs:locatedIn--> town --bs:within--> ... --> wales
the question points the other way, so reverse the last two steps:
bt:place-wales ^bs:within+ ?town ^bs:locatedIn ?shop
-----+----- ------+-------
"everything "the shops in
inside Wales" that town"
Equivalent, and often clearer:
?shop bs:locatedIn/bs:within+ bt:place-wales .
Same answer. Choose whichever reads in the direction you think.
Editor7HOLOS7Fuseki7bookshop-trail-1.1.ttl
Q133
A hop that may not be there
List every place with its council area, falling back to the place itself where there is no council above it.
# Q133 A hop that may not be there
# List every place with its council area, falling back to the place itself where there is no council above it.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?place ?up (COUNT(*) AS ?rows)
WHERE {
?p a bs:Settlement ;
rdfs:label ?place .
?p bs:within? ?u .
?u rdfs:label ?up .
FILTER( LANG(?place) = "en" && LANG(?up) = "en" )
}
GROUP BY ?place ?up
ORDER BY ?place ?up
LIMIT 8
The ? modifier means zero or one hop. It is the path form of OPTIONAL and it behaves quite differently: where OPTIONAL leaves a variable unbound, a zero-length path binds the variable to the starting node. That is useful exactly as often as it is surprising.
What to take away
p? matches zero or one hop, and the zero case binds the variable to the starting node rather than leaving it unbound.
Use OPTIONAL, not p?, when you need to know whether the hop happened. BOUND() has nothing to test after a ? path.
p? and p* both include the zero-length case, so both always produce at least the node itself.
?place bs:within? ?up
Both cases apply to every place, so each one produces TWO rows:
Hay-on-Wye zero hops -> ?up = Hay-on-Wye
Hay-on-Wye one hop -> ?up = Powys
+--------------------+--------------------+
| Aberystwyth | Aberystwyth |
| Aberystwyth | Ceredigion |
| Bath | Bath |
| Bath | Somerset |
+--------------------+--------------------+
That doubling is the thing to expect. A ? path does not choose
between zero and one hop; it matches both, and the row count
doubles for every node that has a parent.
+----------------------------------------------------------+
| bs:within? always binds. Zero hops binds ?up to |
| ?place itself -- never unbound. |
| |
| OPTIONAL { binds or leaves unbound. BOUND(?up) |
| ?place tells them apart; with the path there |
| bs:within ?up is nothing to tell apart. |
| } |
+----------------------------------------------------------+
So ? is the wrong tool when you need to know whether the hop
happened, and the right one when you want "this or its parent"
treated uniformly -- a lookup that should work whether it is
given a town or the county containing it, say.
The four path modifiers, side by side:
p exactly one hop
p? zero or one always binds
p+ one or more
p* zero or more always binds
The two that include zero are the two that always produce a
row, which is why q29 warns that ?x bs:within* ?y matches
every node against itself.
Editor8HOLOS8Fuseki8bookshop-trail-1.1.ttl
Q31
Walking the trail in either direction
Which shops can be reached on foot from The Inkwell?
# Q31 Walking the trail in either direction
# Which shops can be reached on foot from The Inkwell?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?shopName
WHERE {
bt:shop-inkwell (bs:connectsTo|^bs:connectsTo)+ ?shop .
?shop rdfs:label ?shopName .
}
ORDER BY ?shopName
Trail segments are asserted once, from one shop to another, but a footpath is walkable both ways. The alternation (bs:connectsTo|^bs:connectsTo) accepts a step in either direction, and the + around it repeats that step as often as needed.
What to take away
| is alternation: try either path expression at this step.
Combining | with ^ is the standard way to treat a one-way link as two-way, without changing the data.
A + path over an undirected graph makes every connected node reachable from itself. That's correct, and surprising the first time.
asserted: inkwell --connectsTo--> marginalia --> broken-spine
but you can walk it backwards, so the step you want is:
( bs:connectsTo | ^bs:connectsTo )
----+------- -----+--------
forwards backwards
either will do
wrapped in + to repeat:
( bs:connectsTo | ^bs:connectsTo )+
inkwell --> 31 of the 33 shops
(the two south-western shops are on their own; q32)
Note that inkwell reaches ITSELF: go one hop out and one back,
and a + path has found a route home.
# Q32 The shops you can't walk to
# Which shops are cut off from the main trail?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?shopName ?townName
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?shopName ;
bs:locatedIn ?town .
?town rdfs:label ?townName .
FILTER( LANG(?townName) = "en" )
FILTER NOT EXISTS {
bt:shop-inkwell (bs:connectsTo|^bs:connectsTo)+ ?shop .
}
}
ORDER BY ?shopName
Reachability plus negation. NOT EXISTS asks, for each shop, whether any walkable route from The Inkwell arrives there. The two shops in the south west are joined to each other and to nothing else, so no route reaches them.
What to take away
Property paths and NOT EXISTS compose: 'not reachable' is just a reachability test inside a negation.
Connectivity questions are the natural home of + paths.
This is the query to run after adding data, to check nothing has been left stranded.
the main network the south-west spur
inkwell -- ... -- ex-libris west-quay --> penwith
| (joined to each other,
+-- 31 shops reachable and to nothing else)
for each ?shop:
NOT EXISTS { bt:shop-inkwell (bs:connectsTo|^bs:connectsTo)+ ?shop }
|
+-----------------------+
v
no route found -> keep the row
West Quay Books, Penwith Pages
A path that finds nothing is not an error. It is an answer.
Editor2HOLOS2Fuseki2bookshop-trail-1.1.ttl
Q33
Every book of fiction, however narrow the genre
Find all fiction, including books filed under sub-genres several levels down.
# Q33 Every book of fiction, however narrow the genre
# Find all fiction, including books filed under sub-genres several levels down.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
SELECT ?genreName (COUNT(?book) AS ?books)
WHERE {
?book a bs:Work ;
bs:genre ?g .
?g skos:broader* bt:genre-fiction .
?g skos:prefLabel ?genreName .
FILTER( LANG(?genreName) = "en" )
}
GROUP BY ?genreName
ORDER BY DESC(?books) ?genreName
The genre scheme is a SKOS tree of uneven depth: Tartan Noir is three levels below Fiction, Classics is one. skos:broader+ climbs from a book's own genre to every broader concept above it, so a book counts as fiction whichever level it was filed at.
What to take away
skos:broader+ is the standard way to query a subject hierarchy.
Use * rather than + after a step that may already have arrived: here, a book filed directly as Fiction needs zero further climbs.
A path can be built from several steps: bs:genre/skos:broader* is one hop then any number of hops.
Literature
+- Fiction <- the target
+- Crime Fiction
| +- Cosy Crime <- 3 levels down
| +- Tartan Noir <- 3 levels down
+- Speculative Fiction
| +- Science Fiction
| | +- Hard SF <- 4 levels down
| | +- Space Opera
| +- Fantasy
| +- Folk Fantasy
+- Classics <- 2 levels down
?book bs:genre/skos:broader* bt:genre-fiction
-------+----
climb zero or more levels, so a book filed
directly under Fiction still counts
17 concepts sit under Fiction. A two-hop pattern finds 5.
Editor12HOLOS12Fuseki12bookshop-trail-1.1.ttl
Q34
Literary ancestry, and a cycle
Trace every author who influenced Dilys Tremain, directly or at any remove.
# Q34 Literary ancestry, and a cycle
# Trace every author who influenced Dilys Tremain, directly or at any remove.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?ancestorName
WHERE {
bt:author-dilys-tremain bs:influencedBy+ ?ancestor .
?ancestor rdfs:label ?ancestorName .
}
ORDER BY ?ancestorName
bs:influencedBy+ walks the influence graph transitively. The data contains a mutual pair -- two contemporaries who cite each other -- so a naive recursive join would loop forever. A property path will not: the engine tracks which nodes it has already visited.
What to take away
Property paths are cycle-safe. That's a guarantee of the specification, not an accident of one engine.
Transitive closure over a hand-authored graph is where paths pay for themselves.
The answer is a set of ancestors, with no indication of distance. If you need the number of hops, you need something else -- SPARQL has no path-length operator.
dilys-tremain
+-- cerys-lloyd -- elin-morgan -- owain-preece -- bryn-caradoc
| |
| nesta-hywel
| |
| iolo-vaughan
+-- magnus-thole -+- bram-tillotson -- juno-verrall -- ...
+- sandy-cleghorn -- rab-fingal
|
kirsty-lammond
| <- MUTUAL
tam-brodie cycle
bs:influencedBy+ terminates anyway. The path evaluator keeps a
visited set; it is looking for reachable NODES, not for routes.
Write this as a self-join repeated by hand and the cycle hangs
the query.
Editor23HOLOS23Fuseki23bookshop-trail-1.1.ttl
Q35
Everything except the links you name
What does a shop point at, other than its geometry and its stock?
# Q35 Everything except the links you name
# What does a shop point at, other than its geometry and its stock?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX geo: <http://www.opengis.net/ont/geosparql#>
SELECT ?p ?o
WHERE {
bt:shop-colophon !( bs:stocks | geo:hasGeometry | geo:hasDefaultGeometry ) ?o .
bt:shop-colophon ?p ?o .
}
ORDER BY ?p
A negated property set matches any predicate not in the list. It's the way to say 'all the other links', which is useful when exploring, and useful when you want to follow a graph outward without dragging in the bulky parts.
What to take away
!(a|b|c) matches any predicate outside the set.
The negated set is the only place a path may not contain a nested expression -- it takes a plain list of predicates.
Handy for exploring: follow everything except the parts you already understand.
!( bs:stocks | geo:hasGeometry | geo:hasDefaultGeometry )
^ -----------------+----------------------------------
| +-- the predicates to exclude
+-- "any predicate BUT these"
bt:shop-colophon
+- rdf:type ok kept
+- rdfs:label ok kept
+- bs:locatedIn ok kept
+- bs:stocks NO excluded
+- geo:hasGeometry NO excluded
Use ^ inside the set to exclude an incoming link:
!( ^bs:heldAt )
# Q36 A whole journey in one expression
# Name the country of every shop, in a single path.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?shopName ?countryName
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?shopName ;
bs:locatedIn/bs:within+ ?country .
?country a bs:Country ; rdfs:label ?countryName .
FILTER( LANG(?countryName) = "en" )
}
ORDER BY ?countryName ?shopName
The slash builds a sequence: take this step, then that one. Combining a sequence with a repetition gives an expression that reads like the sentence you would say out loud -- a shop is in a town, which is inside some area, which is a country.
What to take away
/ is sequence: one step then the next.
Intermediate nodes in a sequence aren't bound to anything and can't be selected. Split the path if you need them.
Paths make queries shorter, not always clearer. Split a long one when the intermediate steps are part of the answer.
?shop bs:locatedIn / bs:within+ / ^bs:within* ...
-----+----- -----+----
one hop any number
the whole path:
?shop --bs:locatedIn--> town
--bs:within+----> any containing area
|
+- FILTER to keep only countries
reads as: "the shop is in a town, somewhere inside a country"
A sequence with / is evaluated left to right, and the
intermediate nodes are thrown away -- you cannot see the town.
If you need it, use separate patterns.
Editor33HOLOS33Fuseki33bookshop-trail-1.1.ttl
Module 06
Sub-queries
A SELECT inside a WHERE clause. It runs first, produces a small table, and the outer query joins against it. This is how you say 'above average', 'the top three in each group', and 'the one with the most'. A VALUES block is the same shape with the table written by hand instead of computed.
A single query can't compare a row against a summary of all rows, because by the time the aggregate exists the rows are gone. The sub-query computes the average first and yields one row with one column; the outer query then joins every book against that single row and filters.
What to take away
A sub-query runs first and independently; the outer query joins against its result.
This is the only way to compare a value with an aggregate over the same data.
Variables don't leak inwards. A sub-query can't see the outer query's bindings, which is exactly why it can be evaluated once.
+- inner query ----------------------------+
| SELECT (AVG(?p) AS ?avgPrice) |
| WHERE { ?b bs:rrp ?p } |
| |
| result: +----------+ |
| | 15.87 | ONE row |
| +----------+ |
+-------------------+----------------------+
| joined to every outer row
v
+- outer query ----------------------------+
| ?book bs:rrp ?price |
| FILTER( ?price > ?avgPrice ) |
+------------------------------------------+
Inner runs FIRST. It cannot see ?book, ?price or anything else
from the outer query -- only what it computes itself.
Editor15HOLOS15Fuseki15bookshop-trail-1.1.ttl
Q38
The best-attended event at every shop
For each shop, which single event drew the biggest crowd?
# Q38 The best-attended event at every shop
# For each shop, which single event drew the biggest crowd?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?shopName ?eventLabel ?attendance
WHERE {
{
SELECT ?shop (MAX(?a) AS ?attendance)
WHERE { ?e bs:heldAt ?shop ; bs:attendance ?a . }
GROUP BY ?shop
}
?event bs:heldAt ?shop ;
bs:attendance ?attendance ;
rdfs:label ?eventLabel .
?shop rdfs:label ?shopName .
}
ORDER BY DESC(?attendance)
LIMIT 15
Two passes. The inner query finds the maximum attendance per shop, collapsing the events away. The outer query then re-joins that maximum against the events to recover which event it was -- the detail the aggregate had to throw away.
What to take away
Aggregating discards the detail. Joining the aggregate back recovers it.
This shape -- group to find an extreme, re-join to identify it -- is one of the most reusable in SPARQL.
Ties give several rows. If you need exactly one, you must say how to break the tie.
step 1: what is the maximum, per shop?
SELECT ?shop (MAX(?a) AS ?best)
GROUP BY ?shop
+----------+-----+
| ex-libris| 320 |
| endpapers| 210 |
+----------+-----+
|
step 2: join back to find WHICH event that was
?event bs:heldAt ?shop ; bs:attendance ?best
----+
the join condition -------+
+----------+-----+------------------+
| ex-libris| 320 | Launch with Ines |
+----------+-----+------------------+
"Group, then join back" is the standard shape for top-N-per-group.
A tie produces two rows, which is usually what you want.
Editor15HOLOS15Fuseki15bookshop-trail-1.1.ttl
Q39
An aggregate over an aggregate
On average, how many books does each of a publisher's authors write?
# Q39 An aggregate over an aggregate
# On average, how many books does each of a publisher's authors write?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?publisherName (COUNT(?author) AS ?authors)
(ROUND(AVG(?bookCount) * 10) / 10 AS ?meanBooksPerAuthor)
WHERE {
{
SELECT ?publisher ?author (COUNT(?book) AS ?bookCount)
WHERE {
?book bs:publishedBy ?publisher ;
bs:author ?author .
}
GROUP BY ?publisher ?author
}
?publisher rdfs:label ?publisherName .
}
GROUP BY ?publisher ?publisherName
ORDER BY DESC(?meanBooksPerAuthor) ?publisherName
Counting books per author is one aggregation; averaging those counts per publisher is a second, over the results of the first. SPARQL can't nest aggregates in one expression, so the inner grouping has to happen in a sub-query and the outer one over its output.
What to take away
Aggregates don't nest inside one expression; nest the queries instead.
The inner query's grouping keys become ordinary columns to the outer query.
Reading these from the inside out is the only way they make sense.
level 1 -- count books per author per publisher
GROUP BY ?publisher ?author
+-----------+----------------+---+
| northwind | rhona-blackwood| 2 |
| northwind | fenella-drew | 2 |
| northwind | kirsty-lammond | 2 |
+-----------+----------------+---+
|
level 2 -- average those counts per publisher
GROUP BY ?publisher
+-----------+------+---------+
| northwind | 4 | 2.0 |
| |auth. | mean |
+-----------+------+---------+
AVG(COUNT(?x)) is not legal SPARQL. The nesting has to be
expressed as a sub-query, which is the whole reason they exist.
Editor13HOLOS13Fuseki13bookshop-trail-1.1.ttl
Q40
Shops that punch above their weight
Which shops draw a bigger total audience than the average shop does?
# Q40 Shops that punch above their weight
# Which shops draw a bigger total audience than the average shop does?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?shopName ?total ?averageShop
WHERE {
{
SELECT ?shop (SUM(?a) AS ?total)
WHERE { ?e bs:heldAt ?shop ; bs:attendance ?a . }
GROUP BY ?shop
}
{
SELECT (ROUND(AVG(?t) * 10) / 10 AS ?averageShop)
WHERE {
SELECT ?s (SUM(?a) AS ?t)
WHERE { ?e bs:heldAt ?s ; bs:attendance ?a . }
GROUP BY ?s
}
}
?shop rdfs:label ?shopName .
FILTER( ?total > ?averageShop )
}
ORDER BY DESC(?total)
Three levels. The innermost totals attendance per shop; the middle one averages those totals into a single number; the outer query compares each shop against it. Each level is a plain query -- the difficulty is only in seeing which order they run in.
What to take away
Sub-queries nest as deeply as the question needs.
'Average per shop' and 'average per event' are different numbers. Which one you get depends on what you grouped before averaging.
Build these from the inside out, and run each level on its own first.
innermost: total per shop
ex-libris 829, endpapers 520, cotton-quarto 443, ...
|
middle: average of those totals --> 231.4 (one row)
|
outer: keep shops whose total exceeds it
v
+---------------+-------+--------+
| Ex Libris | 829 | 231.4 |
| Endpapers | 520 | 231.4 |
| Cotton Quarto | 443 | 231.4 |
+---------------+-------+--------+
Note the middle query aggregates over the INNER query's rows,
not over the events. Averaging attendance directly would answer
a different question: the average event, not the average shop.
Editor11HOLOS11Fuseki11bookshop-trail-1.1.ttl
Q41
Limiting the inner query, not the outer one
Show every book by the three most prolific authors.
# Q41 Limiting the inner query, not the outer one
# Show every book by the three most prolific authors.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?authorName ?title ?year
WHERE {
{
SELECT ?author (COUNT(?b) AS ?books)
WHERE { ?b a bs:Work ; bs:author ?author . }
GROUP BY ?author
ORDER BY DESC(?books) ?author
LIMIT 3
}
?book bs:author ?author ;
rdfs:label ?title ;
bs:publicationYear ?year .
?author rdfs:label ?authorName .
}
ORDER BY ?authorName ?year
Putting LIMIT in the outer query would cut off books. Putting it in the sub-query picks three authors and then lets the outer query fetch all of their books. The sub-query decides who; the outer query decides what to show.
What to take away
LIMIT inside a sub-query restricts what the outer query joins against, which is a completely different operation from truncating the output.
ORDER BY plus LIMIT inside a sub-query is the idiom for 'the top N of something, then everything about them'.
Ordering an outer query doesn't order its sub-queries, and vice versa.
Add a tie-break. Six authors have three books; without ?author as a second sort key, which three come back is up to the engine, and the three engines really do choose differently.
WRONG RIGHT
?author ... { SELECT ?author
?book bs:author ?author WHERE {...}
LIMIT 3 ORDER BY DESC(?n)
LIMIT 3 }
--> 3 BOOKS ?book bs:author ?author
--> 3 AUTHORS,
all their books
The sub-query is a filter on WHICH authors, evaluated once.
LIMIT inside it limits authors; LIMIT outside limits rows.
ORDER BY inside a sub-query is meaningful precisely because
LIMIT is there to use it.
Editor10HOLOS10Fuseki10bookshop-trail-1.1.ttl
Q42
Authors more prolific than the person who inspired them
Which authors wrote more books than the author who influenced them?
# Q42 Authors more prolific than the person who inspired them
# Which authors wrote more books than the author who influenced them?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?authorName ?ownBooks ?mentorName ?mentorBooks
WHERE {
?author bs:influencedBy ?mentor .
{
SELECT ?author (COUNT(?b) AS ?ownBooks)
WHERE { ?b a bs:Work ; bs:author ?author . }
GROUP BY ?author
}
{
SELECT ?mentor (COUNT(?b2) AS ?mentorBooks)
WHERE { ?b2 a bs:Work ; bs:author ?mentor . }
GROUP BY ?mentor
}
?author rdfs:label ?authorName .
?mentor rdfs:label ?mentorName .
FILTER( ?ownBooks > ?mentorBooks )
}
ORDER BY DESC(?ownBooks) ?authorName
Two independent counts, joined on the influence link. Each sub-query produces a table of author-to-count; the outer pattern joins them through bs:influencedBy so the two counts land in the same row and can be compared.
What to take away
The same sub-query can be used twice with different variables, which is how you compare a thing with a related thing.
The join between the two copies is an ordinary triple pattern in the outer query.
SPARQL has no table aliases; repeating the sub-query is the substitute, and engines are generally clever enough not to compute it twice.
+- count per author -+ +- count per author -+
| dilys-tremain 2 | | cerys-lloyd 2 |
| magnus-thole 2 | | magnus-thole 2 |
+---------+----------+ +---------+----------+
| |
| ?author bs:influencedBy ?mentor
+--------------+--------------+
v
FILTER( ?ownBooks > ?mentorBooks )
The same sub-query appears twice with different variable names.
That is the SPARQL way of aliasing a table -- there is no AS for
a whole sub-query.
Editor8HOLOS8Fuseki8bookshop-trail-1.1.ttl
Q99
A lookup table written into the query
Check three shops against the specialism you expected each to have, with one deliberately left blank.
VALUES can bind several variables at once, which makes it a small join table rather than a list. UNDEF leaves one cell empty, so a row can carry a key with no value -- useful when you want the row to appear even though you have nothing to compare it with.
What to take away
VALUES (?a ?b) { (x y) ... } binds several variables per row, which turns it into a join table rather than a list.
UNDEF leaves a cell unbound. The row survives; the variable is absent, and BOUND() tells them apart.
Comparing stored data against expected values is a test, and this is how you write one without a second dataset.
VALUES (?shop ?expected) {
( bt:shop-inkwell "Crime Fiction" )
( bt:shop-marginalia "Poetry" )
( bt:shop-errata UNDEF )
}
a two-column table, joined on ?shop:
+--------------------+-----------------+
| ?shop | ?expected |
+--------------------+-----------------+
| bt:shop-inkwell | "Crime Fiction" |
| bt:shop-marginalia | "Poetry" |
| bt:shop-errata | (unbound) | <- UNDEF
+--------------------+-----------------+
UNDEF is not the empty string and not zero. The row is kept and
the variable is simply not bound, exactly as if an OPTIONAL had
failed -- so BOUND() is how you test for it, and the comparison
below reports "not checked" rather than a mismatch.
A parenthesised VALUES with n variables is n columns wide; every
row must have n entries, and UNDEF is how you leave one out.
Editor3HOLOS3Fuseki3bookshop-trail-1.1.ttl
Module 07
Other query forms
SELECT isn't the only answer shape. CONSTRUCT builds a new graph, ASK returns a boolean, DESCRIBE hands back whatever the engine thinks describes a resource. This module introduces the three; module 12 uses them in earnest, and is worth reaching for as soon as these five make sense.
# Q43 Build a simpler graph
# Produce a small graph of shops and the names of their towns, ready to paste back into the editor.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
CONSTRUCT {
?shop rdfs:label ?shopName ;
bs:townName ?townName .
}
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?shopName ;
bs:locatedIn ?town .
?town rdfs:label ?townName .
FILTER( LANG(?townName) = "en" )
}
CONSTRUCT returns RDF instead of a table. The template between the braces is filled in once per solution, so the output is a graph whose shape you chose rather than the one the data happens to have.
What to take away
CONSTRUCT returns a graph. Its template is instantiated once per solution.
It's the standard way to reshape data for another tool, or to simplify before visualising.
A graph is a set, so duplicate triples collapse once the result is loaded somewhere. The result stream itself is another matter -- see q44, where the three engines disagree about exactly that.
WHERE finds solutions CONSTRUCT builds triples
?shop = bt:shop-inkwell bt:shop-inkwell
?name = "The Inkwell" --> rdfs:label "The Inkwell" ;
?town = "Wigtown" bs:townName "Wigtown" .
one solution -------------> two triples
The template may invent predicates that appear nowhere in the
source: bs:townName is created here, purely for the output.
In the Turtle Editor Viewer the result comes back as Turtle --
paste it into the editor pane and the graph view will draw it.
Editor66HOLOS66Fuseki66bookshop-trail-1.1.ttl
Q44
Materialise the links that aren't there
The data records bs:imprintOf but never bs:hasImprint. Create it.
CONSTRUCT can assert the inverse of an existing link, turning a query into a small inference step. The result is a graph you can load alongside the original -- which is what an OWL reasoner would do for you, more slowly and with more ceremony.
What to take away
CONSTRUCT is the cheapest form of inference: you state the rule as a query.
The output is a graph, so it can be loaded back and queried alongside the source.
^bs:imprintOf answers the same question without materialising anything. Materialise only when many queries will need it.
Count what comes back before you trust it. This template emits three triples per solution, and several solutions share a parent, so the same label triple is built more than once.
in the data:
pub-saltmarsh --bs:imprintOf--> pub-northwind
CONSTRUCT { ?parent bs:hasImprint ?child }
WHERE { ?child bs:imprintOf ?parent }
produces:
pub-northwind --bs:hasImprint--> pub-saltmarsh
The vocabulary already declares
bs:hasImprint owl:inverseOf bs:imprintOf
so a reasoner would infer this. CONSTRUCT does the same job in
one query, with no reasoner and no surprises.
In the editor, 'Show Facts' runs the HyLAR reasoner and infers
it from the OWL declaration instead. Compare the two.
Engines differ, and that's expected. 27 triples from the browser editor, 22 from HOLOS and Fuseki. The template is instantiated nine times, giving 27 triples of which 22 are distinct. HOLOS and Fuseki return the set; Comunica returns the stream, duplicates and all. Both are defensible -- a graph is a set, but a result stream need not be -- and it matters the moment you count rows instead of loading them.
# Q45 A yes or no question
# Is there a bookshop in Wales with a cafe?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
ASK {
?shop bs:locatedIn/bs:within+ bt:place-wales ;
bs:hasCafe true .
}
ASK returns a single boolean. The engine may stop at the first match, so it's the cheapest way to test for existence -- and much better than running a SELECT and counting the rows yourself.
What to take away
ASK answers existence questions and returns one boolean.
It's cheaper than SELECT because the engine can stop at the first solution.
Use it for tests -- in a script, or in a validation step -- not for fetching data.
ASK {
?shop bs:locatedIn/bs:within+ bt:place-wales ;
bs:hasCafe true .
}
+-------------+
| any match? |
+------+------+
|
+--------+--------+
v v
true false
One value comes back, not a table. The engine is allowed to
stop looking the moment it finds one solution.
In the editor the result appears as a single true/false rather
than a results grid.
DESCRIBE hands back a graph the engine thinks describes the resource. What counts as a description is up to the engine, which makes DESCRIBE convenient for exploring and unwise to rely on in code that must behave identically everywhere.
What to take away
DESCRIBE returns a graph chosen by the engine, and the specification deliberately leaves the choice open.
It's excellent for exploring an unfamiliar endpoint.
Never build a pipeline on it: write the CONSTRUCT you mean instead.
DESCRIBE bt:shop-quire
most engines return the "concise bounded description":
bt:shop-quire ?p ?o <- all outgoing statements
plus, for any ?o that is a blank node,
that node's statements too, recursively
what you get is NOT specified:
Jena outgoing statements
HOLOS outgoing statements
Comunica outgoing statements
another engine might include incoming ones too
For anything reproducible, write the CONSTRUCT you actually
mean. DESCRIBE is for looking around.
Engines differ, and that's expected. and that's the lesson. Measured on this dataset, the three engines return different graphs for the same DESCRIBE. Nothing is broken -- the specification leaves the choice to the engine. This is the only query in the course whose answer is allowed to vary, and the only one where that's the point.
Editor18HOLOS18Fuseki18bookshop-trail-1.1.ttl
Q47
A summary graph worth keeping
Build a compact profile of every shop: name, town, country, specialism and event count.
CONSTRUCT with aggregation. The sub-query counts events per shop; the template then assembles one tidy node per shop. The output is small enough to paste into the editor and see whole, which the source data isn't.
What to take away
CONSTRUCT and aggregation combine: compute in the WHERE, assemble in the template.
Producing a small summary graph is the practical answer to 'this dataset is too big to visualise'.
The output is valid Turtle. Save it, load it, query it again.
5,000-triple dataset ~165-triple summary
+--------------------+ +--------------------+
| shops, towns, | --> | bt:shop-quire |
| councils, regions, | | rdfs:label ... |
| countries, events, | | bs:townName ... |
| geometry, stock... | | bs:countryName ..|
+--------------------+ | bs:eventCount 2 |
+--------------------+
This is the query to run before visualising. The editor's graph
view draws 10 subjects at a time; a summary makes those 10
subjects worth looking at.
Editor165HOLOS165Fuseki165bookshop-trail-1.1.ttl
Module 08
Named graphs and federation
The dataset also ships as TriG, with each subject area in its own named graph. GRAPH lets you ask where a fact came from, which is the cheapest form of provenance there's. The second half of the module takes the same idea across the network: SERVICE puts the other graph on somebody else's machine, and the last four queries join this dataset to DBpedia.
# Q48 What graphs are in this dataset
# The TriG file splits the data by subject matter. What are the parts called, and how big is each?
# Data: bookshop-trail.trig
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
SELECT ?g (COUNT(*) AS ?triples)
WHERE {
GRAPH ?g { ?s ?p ?o }
}
GROUP BY ?g
ORDER BY DESC(?triples)
GRAPH ?g binds the name of the graph a pattern matched in. With the pattern left completely open, the result is an inventory of the dataset -- the first thing worth asking of an endpoint you haven't seen before.
What to take away
GRAPH ?g { ... } binds ?g to the name of the graph the pattern matched in.
The default graph isn't one of the named graphs and won't be found this way.
Counting triples per graph is the fastest way to understand an unfamiliar dataset's layout.
a dataset is a default graph plus zero or more named graphs:
+ default graph --------------------------+
| bt:dataset rdfs:label "The Bookshop..."|
+-----------------------------------------+
+ bt:graph-places -+ + bt:graph-books -+
| 30 settlements | | 74 works |
| 23 councils ... | | ... |
+------------------+ +-----------------+
GRAPH ?g { ?s ?p ?o }
|
+-- binds to the NAME of whichever graph matched
Triples in the default graph are NOT visible to GRAPH ?g.
That catches people out: the dataset description in the default
graph does not appear in this count.
Editor10HOLOS10Fuseki10bookshop-trail.trig
Q49
Where did this fact come from
Which part of the dataset asserts each thing known about The Inkwell?
# Q49 Where did this fact come from
# Which part of the dataset asserts each thing known about The Inkwell?
# Data: bookshop-trail.trig
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
SELECT ?g ?p ?o
WHERE {
GRAPH ?g { bt:shop-inkwell ?p ?o }
}
ORDER BY ?g ?p
The same pattern as an ordinary query, wrapped in GRAPH ?g. Every solution now carries the name of the graph that supplied it, which is provenance at its cheapest -- no extra vocabulary, no annotation, just the filing system.
What to take away
Wrapping a pattern in GRAPH ?g adds the source to every row.
This is the cheapest provenance mechanism there's, and often enough.
It records where a fact is filed, not who asserted it. RDF 1.2 annotations answer the second question.
bt:shop-inkwell ?p ?o
without GRAPH: where did that come from? no idea
GRAPH ?g { bt:shop-inkwell ?p ?o }
?g = bt:graph-shops ?p = bs:founded ?o = 1979
?g = bt:graph-shops ?p = bs:hasCafe ?o = true
?g = bt:graph-stock ?p = bs:stocks ?o = bt:book-...
?g = bt:graph-trail ?p = bs:connectsTo ?o = bt:shop-...
|
+-- the same subject, facts from three graphs
Named graphs give you per-triple provenance for free, as long as
"which file it came from" is the granularity you need. For finer
grain -- who said it, when, how sure -- see module 11.
Editor23HOLOS23Fuseki23bookshop-trail.trig
Q50
Querying one graph, then all of them
Count the shops using only the shops graph, and then across the whole dataset.
# Q50 Querying one graph, then all of them
# Count the shops using only the shops graph, and then across the whole dataset.
# Data: bookshop-trail.trig
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
SELECT ?scope (COUNT(?s) AS ?shops)
WHERE {
{
GRAPH bt:graph-shops { ?s a bs:Bookshop }
BIND( "inside GRAPH bt:graph-shops" AS ?scope )
}
UNION
{
?s a bs:Bookshop .
BIND( "default graph, no GRAPH keyword" AS ?scope )
}
UNION
{
GRAPH ?any { ?s a bs:Bookshop }
BIND( "any named graph" AS ?scope )
}
}
GROUP BY ?scope
ORDER BY ?scope
PREFIXbt:<https://example.org/bookshop-trail/>PREFIXbs:<https://example.org/bookshop-trail/schema#>SELECT?scope (COUNT(?s) AS?shops)
WHERE {
{
GRAPHbt:graph-shops { ?sabs:Bookshop }
BIND( "inside GRAPH bt:graph-shops"AS?scope )
}
UNION
{
?sabs:Bookshop .
BIND( "default graph, no GRAPH keyword"AS?scope )
}
UNION
{
GRAPH?any { ?sabs:Bookshop }
BIND( "any named graph"AS?scope )
}
}
GROUPBY?scopeORDERBY?scope
How it works
A pattern inside GRAPH sees only that graph. The same pattern outside GRAPH sees the default graph, which in this TriG file holds nothing but the dataset description -- so it finds nothing at all. That surprise is the lesson.
What to take away
A pattern not inside GRAPH matches the default graph only.
Loading TriG instead of Turtle can silently change what your query returns.
Many endpoints put everything in the default graph as well. Never assume; run this query and find out.
GRAPH bt:graph-shops { ?s a bs:Bookshop } --> 33
{ ?s a bs:Bookshop } (no GRAPH) --> 0
^
because the default graph of this TriG file |
contains only the dataset description ---------+
Compare with the Turtle files, where everything IS the default
graph and the second pattern finds all 33.
Same triples, different filing, different answers. Know which
kind of file you loaded.
Editor2HOLOS2Fuseki2bookshop-trail.trig
Q51
Joining across two graphs
Pair each shop with its town's name, when the shops and the places live in different graphs.
# Q51 Joining across two graphs
# Pair each shop with its town's name, when the shops and the places live in different graphs.
# Data: bookshop-trail.trig
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?shopName ?townName
WHERE {
GRAPH bt:graph-shops {
?shop a bs:Bookshop ;
rdfs:label ?shopName ;
bs:locatedIn ?town .
}
GRAPH bt:graph-places {
?town rdfs:label ?townName .
}
FILTER( LANG(?townName) = "en" )
}
ORDER BY ?townName ?shopName
LIMIT 20
One GRAPH block per source, joined on the shared variable exactly as ordinary patterns are. Nothing about the join changes because the data is filed separately -- which is the point of named graphs.
What to take away
Patterns in different GRAPH blocks join on shared variables like any others.
Named graphs partition storage; they don't partition querying.
The same shape scales up to SERVICE, where the other graph is on another machine.
+ bt:graph-shops ------------+
| ?shop bs:locatedIn ?town |--+
+----------------------------+ | joined on ?town
+ bt:graph-places -----------+ |
| ?town rdfs:label ?townName |<-+
+----------------------------+
The join is the shared variable, as always. Graph boundaries do
not obstruct it.
This is how federated queries work too: SERVICE replaces GRAPH,
and the other side is a different server rather than a different
graph.
Editor20HOLOS20Fuseki20bookshop-trail.trig
Q124
Choosing the dataset in the query
Answer a question against two of the ten graphs and ignore the rest.
# Q124 Choosing the dataset in the query
# Answer a question against two of the ten graphs and ignore the rest.
# Data: bookshop-trail.trig
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?name ?town
FROM bt:graph-shops
FROM bt:graph-places
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name ;
bs:locatedIn ?place .
?place rdfs:label ?town .
FILTER( LANG(?town) = "en" )
}
ORDER BY ?name
LIMIT 8
FROM names the graphs to merge into the default graph for this query only. Nothing is copied and nothing is changed: the engine builds a dataset for the duration of the query, and the ordinary triple patterns see exactly that. It is the cheapest way to scope a question.
What to take away
FROM builds this query's default graph by merging the graphs it names. It reads; it does not copy or change anything.
Several FROM clauses merge into one graph. Use FROM NAMED when you need to know which graph a fact came from.
Without a dataset clause you get whatever the service calls the default graph, and that is not standardised.
the store this query's dataset
--------- --------------------
bt:graph-vocabulary
bt:graph-genres
bt:graph-places ------ bt:graph-shops ---\ bt:graph-people \ '--> default graph
bt:graph-books '-----> (places + shops merged)
bt:graph-events
bt:graph-trail everything else: invisible
bt:graph-stock
bt:graph-claims
FROM bt:graph-shops
FROM bt:graph-places
Two clauses, one default graph. FROM does not give you two
graphs you can tell apart -- for that you want FROM NAMED and
GRAPH, which is q125.
+----------------------------------------------------------+
| no FROM at all the service decides what the default |
| graph is, and services disagree. On |
| this TriG file the default graph holds |
| only the descriptions of the graphs, |
| so ?s a bs:Bookshop finds nothing. |
| That is q50's surprise. |
+----------------------------------------------------------+
A caution worth carrying: FROM takes an IRI, and an engine
that does not hold that graph may go and fetch it over HTTP.
Same exposure as SERVICE and LOAD.
Editor8HOLOS8Fuseki8bookshop-trail.trig
Q125
Keeping the graphs apart
Count the triples in two named graphs, and say which is which.
# Q125 Keeping the graphs apart
# Count the triples in two named graphs, and say which is which.
# Data: bookshop-trail.trig
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
SELECT ?graph (COUNT(*) AS ?triples)
FROM NAMED bt:graph-shops
FROM NAMED bt:graph-places
WHERE {
GRAPH ?graph { ?s ?p ?o }
}
GROUP BY ?graph
ORDER BY ?graph
FROM NAMED adds a graph to the dataset without merging it, so GRAPH can still name it. This is the pairing that gives you provenance: FROM for the facts you want to treat as one body, FROM NAMED for the ones whose origin matters.
What to take away
FROM NAMED puts a graph in the dataset without merging it, so GRAPH can still ask which graph a fact is in.
FROM and FROM NAMED are independent. A query can have both, and a graph named only by FROM NAMED is not in the default graph.
The browser editor does not support FROM NAMED. Scope with FROM there, and check the dataset clauses on the engine you will deploy against.
FROM merges into the default graph -- origin lost
FROM NAMED keeps the graph addressable -- origin kept
FROM NAMED bt:graph-shops
FROM NAMED bt:graph-places
WHERE { GRAPH ?g { ?s ?p ?o } }
+-------------------------+-------+
| bt:graph-places | 794 |
| bt:graph-shops | 588 |
+-------------------------+-------+
and the eight other graphs contribute nothing, because they
are not in this query's dataset at all.
measured:
FROM editor yes holos yes fuseki yes
FROM NAMED editor ERROR holos yes fuseki yes
Comunica raises rather than answering: over an in-memory store
it has no actor for the pattern a FROM NAMED dataset produces.
So in the browser, scope with FROM and load the TriG file; keep
FROM NAMED for a real endpoint.
Note. Fuseki and HOLOS. Comunica raises "none of the configured actors were able to handle the operation type pattern" for FROM NAMED over an in-memory store; FROM on its own (q124) works there.
EditorHOLOS2Fuseki2bookshop-trail.trig
Q105
Bringing in DBpedia
How many people live in each of the three book towns, according to DBpedia?
# Q105 Bringing in DBpedia
# How many people live in each of the three book towns, according to DBpedia?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX dbo: <http://dbpedia.org/ontology/>
SELECT ?name ?population
WHERE {
?town a bs:Settlement ;
bs:isBookTown true ;
rdfs:label ?name ;
owl:sameAs ?dbp .
FILTER( LANG(?name) = "en" )
SERVICE <https://dbpedia.org/sparql> {
?dbp dbo:populationTotal ?population .
}
}
ORDER BY ?name
SERVICE sends part of the query to another endpoint and joins the answer back. The join key is owl:sameAs: the shops are invented but the towns are real, and each one carries a link to its DBpedia resource, so the remote side is asked about a resource rather than a name.
What to take away
SERVICE evaluates a pattern at another endpoint and joins the result into the surrounding query.
Federate on identifiers, not on labels. owl:sameAs is the link that makes two datasets about the same thing joinable.
The remote side is someone else's data, with its own gaps. A join that drops rows is usually telling you about their coverage, not your query.
here dbpedia.org
---- -----------
?town a bs:Settlement ;
bs:isBookTown true ;
rdfs:label ?name ;
owl:sameAs ?dbp -------------> ?dbp dbo:populationTotal ?population
SERVICE
| |
'--------------- joined on ?dbp ---------'
what comes back:
Sedbergh 2765
Hay-on-Wye (DBpedia has no populationTotal for it)
Wigtown (nor for this one)
So the join drops two of the three -- and that is not a bug in
the query, it is what the remote data is like. q106 is about
getting them back.
owl:sameAs is what makes this work. Matching "Sedbergh" the
string against a remote label would find the right town by luck;
matching the IRI finds it by identity.
Note. HOLOS refuses remote SERVICE outright -- see q108, which is about why. On the other two this returns one row, because DBpedia holds a population for Sedbergh and not for the other two book towns.
Editor1HOLOSFuseki1bookshop-trail-1.1.ttl
Q106
Keeping the rows the remote side cannot answer
List all three book towns, with the population where DBpedia has one.
# Q106 Keeping the rows the remote side cannot answer
# List all three book towns, with the population where DBpedia has one.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX dbo: <http://dbpedia.org/ontology/>
SELECT ?name ?population
WHERE {
?town a bs:Settlement ;
bs:isBookTown true ;
rdfs:label ?name ;
owl:sameAs ?dbp .
FILTER( LANG(?name) = "en" )
OPTIONAL {
SERVICE <https://dbpedia.org/sparql> {
?dbp dbo:populationTotal ?population .
}
}
}
ORDER BY ?name
The obvious fix for q105 dropping rows is to wrap the SERVICE in an OPTIONAL, and on Jena that is exactly right. In the browser editor it returns all three rows and no populations at all: Comunica does not push the outer binding into a SERVICE nested inside an OPTIONAL. Same query, same endpoint, different answer.
What to take away
OPTIONAL around SERVICE is the usual way to keep rows the remote side cannot answer, and it is not portable.
Check a federated query on the engine you will actually run it on. The row count agreeing proves nothing.
When it does not work, push the values in explicitly instead -- q107.
OPTIONAL {
SERVICE <https://dbpedia.org/sparql> {
?dbp dbo:populationTotal ?population
}
}
measured, this dataset, this endpoint:
+---------+-------+---------------------------------------+
| Fuseki | 3 rows| Sedbergh 2765, the other two blank ok |
| editor | 3 rows| ALL THREE blank |
| HOLOS | -- | refuses remote SERVICE entirely |
+---------+-------+---------------------------------------+
Comunica gets the row count right and the data wrong, which is
the hardest kind of difference to notice.
Without the OPTIONAL (q105) both engines agree, because then the
binding goes in as part of an ordinary join. The disagreement is
specifically about OPTIONAL wrapping SERVICE.
q107 is the shape that works on both.
Engines differ, and that's expected. and the difference is the lesson. Fuseki fills in Sedbergh's population; the browser editor returns the same three rows with the column empty, because Comunica does not carry the outer binding into a SERVICE inside an OPTIONAL.
Editor3HOLOSFuseki3bookshop-trail-1.1.ttl
Q107
Sending the list with the question
Ask DBpedia about a batch of towns in one round trip, in a way both engines answer.
# Q107 Sending the list with the question
# Ask DBpedia about a batch of towns in one round trip, in a way both engines answer.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX dbo: <http://dbpedia.org/ontology/>
SELECT ?dbp ?population
WHERE {
SERVICE <https://dbpedia.org/sparql> {
VALUES ?dbp {
<http://dbpedia.org/resource/York>
<http://dbpedia.org/resource/Bath,_Somerset>
<http://dbpedia.org/resource/Penzance>
<http://dbpedia.org/resource/Sedbergh>
<http://dbpedia.org/resource/Norwich>
}
?dbp dbo:populationTotal ?population .
}
}
ORDER BY DESC(?population)
Rather than relying on the engine to push bindings across, put them in the SERVICE block yourself with VALUES. The remote endpoint gets one self-contained query naming exactly the resources you care about, and there is nothing for an engine to get wrong.
What to take away
VALUES inside a SERVICE block sends the keys with the question, which is the most portable way to federate.
One round trip for n resources beats n round trips. Federation is dominated by latency, not by matching.
A self-contained SERVICE block can be tested directly against the remote endpoint, which is how you tell whose fault an empty result is.
SERVICE <https://dbpedia.org/sparql> {
VALUES ?dbp {
<http://dbpedia.org/resource/York>
<http://dbpedia.org/resource/Bath,_Somerset>
...
}
?dbp dbo:populationTotal ?population .
}
five IRIs go out, four answers come back:
+-------------------------+------------+
| York | 141,685 |
| Bath, Somerset | 94,092 |
| Penzance | 20,734 |
| Sedbergh | 2,765 |
| Norwich | -- no dbo:populationTotal --
+-------------------------+------------+
Norwich drops out for the same reason Hay-on-Wye did in q105.
Asking for five things and getting four is normal when the data
is somebody else's.
why this is the shape to reach for:
- both engines answer it identically
- one round trip rather than one per row
- the remote endpoint sees a query it can plan properly
- you can paste the SERVICE block straight into DBpedia's own
form to see what it does on its own
The cost is that the list is fixed. Generate the VALUES block
from a first, local query when it needs to vary.
Editor4HOLOSFuseki4bookshop-trail-1.1.ttl
Q108
When the other end is not there
What happens when the remote endpoint is unreachable, and why does HOLOS refuse to call one at all?
# Q108 When the other end is not there
# What happens when the remote endpoint is unreachable, and why does HOLOS refuse to call one at all?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?name ?remote
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name .
# The host does not exist. SILENT is what decides whether that
# ends the query or is simply shrugged off.
SERVICE SILENT <https://endpoint.invalid/sparql> {
?shop bs:somethingRemote ?remote .
}
}
ORDER BY ?name
LIMIT 5
PREFIXbs:<https://example.org/bookshop-trail/schema#>PREFIXrdfs:<http://www.w3.org/2000/01/rdf-schema#>SELECT?name?remoteWHERE {
?shopabs:Bookshop ;
rdfs:label?name .
# The host does not exist. SILENT is what decides whether that# ends the query or is simply shrugged off.SERVICE SILENT <https://endpoint.invalid/sparql> {
?shopbs:somethingRemote?remote .
}
}
ORDERBY?nameLIMIT5
How it works
SERVICE SILENT tells the engine to carry on with no bindings rather than fail when a remote call goes wrong. Fuseki honours it. The browser editor raises anyway. HOLOS declines to make the request in the first place, and its reason is worth understanding.
What to take away
SERVICE SILENT continues with no bindings instead of failing. Fuseki honours it; Comunica raises anyway, so do not rely on it in the browser.
A public endpoint that follows arbitrary SERVICE IRIs will make requests to any address a stranger names, including ones only it can reach. That is SSRF, and it is why HOLOS refuses.
Federation is a network operation wearing the clothes of a join: it can be slow, it can fail, and it can be a security question.
SERVICE SILENT <https://endpoint.invalid/sparql> { ... }
measured against a host that does not exist:
+---------+------------------+--------------------------+
| | SERVICE | SERVICE SILENT |
+---------+------------------+--------------------------+
| Fuseki | error | 3 rows, no bindings ok |
| editor | error | error |
| HOLOS | refuses | refuses |
+---------+------------------+--------------------------+
WHY HOLOS REFUSES
A SERVICE IRI is a URL chosen by whoever wrote the query. An
engine that follows it will make its server issue a request to
that address -- from inside your network:
SERVICE <http://169.254.169.254/latest/meta-data/>
SERVICE <http://localhost:9200/>
SERVICE <http://admin.internal/>
That is server-side request forgery, and the query language is
the attack surface. HOLOS evaluates SERVICE only against
endpoints registered in the process, and refuses remote HTTP
outright; it refuses remote LOAD for the same reason. Enabling
it safely needs an allow-list, which is a policy decision rather
than a default.
Not a bug, then, but a position. Worth knowing which position
your own endpoint takes before you expose it.
Note. Fuseki only. The browser editor raises rather than honouring SILENT, and HOLOS refuses remote SERVICE altogether -- which is the point of the query rather than a limitation of it.
EditorHOLOSFuseki5bookshop-trail-1.1.ttl
Module 09
Geospatial with nothing but arithmetic
Every engine can do geography if the coordinates are plain numbers. This module builds bounding boxes and a great-circle distance out of FILTER and BIND alone -- so it runs in the browser, with no GeoSPARQL support of any kind.
# Q52 Shops in a box on the map
# Which shops lie in the south west, between 50 and 52 degrees north and west of 2 degrees?
# Data: bookshop-trail-full.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX wgs84: <http://www.w3.org/2003/01/geo/wgs84_pos#>
SELECT ?name ?lat ?long
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name ;
wgs84:lat ?lat ;
wgs84:long ?long .
FILTER( ?lat > 50.0 && ?lat < 52.0 && ?long < -2.0 )
}
ORDER BY ?lat
A bounding box is four comparisons on two numbers. The dataset publishes plain wgs84:lat and wgs84:long decimals next to the WKT geometry precisely so that this works in an engine with no geospatial support at all.
What to take away
A bounding box is the cheapest spatial filter there's, and needs nothing beyond numeric comparison.
Publishing coordinates as plain decimals alongside the WKT costs a few triples and makes the data usable in engines with no geospatial support.
Use a box to prefilter before an exact test. The box is wrong at the corners, but it's fast, and the exact test then fixes the corners.
long -6 -2
| |
52 --+--------------+-- lat 52
| *Aberystwyth|
| *Bath | * inside the box -> kept
| *Exeter | o outside -> dropped
|*Penzance |
50 --+--------------+-- lat 50
| |
FILTER( ?lat > 50 && ?lat < 52 && ?long < -2 )
Four numeric comparisons. No functions, no extensions, no
GeoSPARQL. This runs in the browser editor unchanged.
A box is also the right FIRST step for an expensive query: cheap
to evaluate, and it throws away most of the candidates before
anything costly runs.
Ranking by distance doesn't need the distance. If A is nearer than B then A's squared distance is smaller too, so sorting on the square gives exactly the right order -- and squaring needs only multiplication, which SPARQL 1.1 does have. Degrees of longitude are shorter than degrees of latitude, so each is scaled to kilometres before squaring.
What to take away
To rank by distance you never need the square root. This is the single most useful trick for geography in a plain SPARQL engine.
A degree of longitude isn't a degree of latitude. Scale them separately or every east-west distance is overstated.
Know the error in your approximation before you rely on it, and say what it's.
SPARQL 1.1 has: + - * / ABS ROUND FLOOR CEIL
SPARQL 1.1 lacks: sin cos tan sqrt pow atan2
So a real great-circle distance is out of reach. But:
sqrt(x) is monotonic => ordering by x
== ordering by sqrt(x)
+-- scale degrees to kilometres -----------------+
| 1 deg latitude ~ 111.19 km (everywhere) |
| 1 deg longitude ~ 66.70 km (at 53 N) |
+------------------------------------------------+
dy = (?lat - 52.0760) * 111.19
dx = (?long + 3.1288 ) * 66.70
?d2 = dx*dx + dy*dy <- squared km, never rooted
ORDER BY ?d2 gives the true nearest-first order.
The approximation, measured: against a proper
haversine over all 435 pairs of settlements in this dataset, the
median error is 0.3%, the 95th percentile 4.1%, and the worst
case 11.4% -- Inverness to Portree, where 57 N is a long way from
the 53 N the longitude scale assumes. q55 shows that failure,
and q54 avoids it.
Editor12HOLOS12Fuseki12bookshop-trail-full.ttl
Q54
Exact distances, by using a flat map
Which shop is nearest to each of the four towns that have none?
The approximation in q53 exists only because degrees aren't a length. A projected coordinate system fixes that at the source: the British National Grid is already flat and already in metres, so Pythagoras on eastings and northings is simply correct. The dataset publishes both, and this query uses the grid.
What to take away
Projecting the data once removes the need to approximate in every query. If you do geography often, store a projected coordinate.
The same squared-distance trick applies, and here it's exact rather than approximate.
Nearest-neighbour is 'minimum per group, then join back to find which one'.
degrees British National Grid
------- ---------------------
lat/long on a sphere eastings/northings on a plane
a degree is not a length the unit IS the metre
scale factor varies with scale error across Britain
latitude is under 0.04%
dx = ?e1 - ?e2 (metres, exactly)
dy = ?n1 - ?n2
d2 = dx*dx + dy*dy (metres squared)
for each town with no shop:
Durham --> nearest shop
Perth --> nearest shop
Fort William --> nearest shop
Truro --> nearest shop
The inner query finds the minimum squared distance per town; the
outer one joins back to discover which shop that was -- the same
"group, then join back" shape as q38.
Editor4HOLOS4Fuseki4bookshop-trail-full.ttl
Q55
Where the flat-Earth shortcut breaks
Show a pair of places where degree arithmetic and grid arithmetic disagree.
# Q55 Where the flat-Earth shortcut breaks
# Show a pair of places where degree arithmetic and grid arithmetic disagree.
# Data: bookshop-trail-full.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX wgs84: <http://www.w3.org/2003/01/geo/wgs84_pos#>
SELECT ?fromName ?toName ?gridKm2 ?degreeKm2 ?ratio
WHERE {
?a a bs:Settlement ; rdfs:label ?fromName ;
bs:easting ?ae ; bs:northing ?an ; wgs84:lat ?alat ; wgs84:long ?alon .
?b a bs:Settlement ; rdfs:label ?toName ;
bs:easting ?be ; bs:northing ?bn ; wgs84:lat ?blat ; wgs84:long ?blon .
FILTER( STR(?fromName) < STR(?toName) )
FILTER( LANG(?fromName) = "en" && LANG(?toName) = "en" )
BIND( (?be - ?ae) * (?be - ?ae) + (?bn - ?an) * (?bn - ?an) AS ?gridM2 )
BIND( ((?blat - ?alat) * 111.19) AS ?dy )
BIND( ((?blon - ?alon) * 66.70) AS ?dx )
BIND( ?dx * ?dx + ?dy * ?dy AS ?degKm2 )
FILTER( ?gridM2 > 100000000 )
BIND( ROUND(?gridM2 / 1000000.0) AS ?gridKm2 )
BIND( ROUND(?degKm2) AS ?degreeKm2 )
# Divide the two small rounded values, not the raw squared metres. Dividing
# by a ten-digit decimal expression inline left ?ratio unbound on one of the
# three engines -- and an unbound sort key means ORDER BY has nothing to
# work with, so the "worst" rows were not the worst at all.
BIND( ROUND(?degreeKm2 * 1000.0 / ?gridKm2) / 1000.0 AS ?ratio )
}
ORDER BY DESC(?ratio)
LIMIT 8
PREFIXbs:<https://example.org/bookshop-trail/schema#>PREFIXrdfs:<http://www.w3.org/2000/01/rdf-schema#>PREFIXwgs84:<http://www.w3.org/2003/01/geo/wgs84_pos#>SELECT?fromName?toName?gridKm2?degreeKm2?ratioWHERE {
?aabs:Settlement ; rdfs:label?fromName ;
bs:easting?ae ; bs:northing?an ; wgs84:lat?alat ; wgs84:long?alon .
?babs:Settlement ; rdfs:label?toName ;
bs:easting?be ; bs:northing?bn ; wgs84:lat?blat ; wgs84:long?blon .
FILTER( STR(?fromName) < STR(?toName) )
FILTER( LANG(?fromName) = "en" && LANG(?toName) = "en" )
BIND( (?be - ?ae) * (?be - ?ae) + (?bn - ?an) * (?bn - ?an) AS?gridM2 )
BIND( ((?blat - ?alat) * 111.19) AS?dy )
BIND( ((?blon - ?alon) * 66.70) AS?dx )
BIND( ?dx * ?dx + ?dy * ?dyAS?degKm2 )
FILTER( ?gridM2 > 100000000 )
BIND( ROUND(?gridM2 / 1000000.0) AS?gridKm2 )
BIND( ROUND(?degKm2) AS?degreeKm2 )
# Divide the two small rounded values, not the raw squared metres. Dividing# by a ten-digit decimal expression inline left ?ratio unbound on one of the# three engines -- and an unbound sort key means ORDER BY has nothing to# work with, so the "worst" rows were not the worst at all.BIND( ROUND(?degreeKm2 * 1000.0 / ?gridKm2) / 1000.0AS?ratio )
}
ORDERBYDESC(?ratio)
LIMIT8
How it works
Both metrics are computed for every pair of settlements and their ratio taken. Near 53 degrees north the two agree closely; at the top of Scotland the fixed longitude scale is badly wrong, and the degree-based figure overstates the distance by more than a tenth.
What to take away
An approximation that's fine in the middle of your data can be badly wrong at its edges.
East-west distances at high latitude are where a fixed longitude scale fails first.
Comparing two methods on the same data is the cheapest way to find out whether the simpler one is good enough.
Keep intermediate values small and give them names. Dividing by a ten-digit expression inline left ?ratio unbound on one of the three engines; dividing the two rounded kilometre figures instead works everywhere and reads better.
An unbound ORDER BY key doesn't raise an error. It just stops sorting, and the top of your result is then whatever the engine happened to produce first.
degree metric assumes: 1 deg longitude = 66.70 km (true at 53 N)
but a degree of longitude shrinks towards the pole:
at 50 N -> 71.7 km degree metric UNDERSTATES
at 53 N -> 66.9 km about right
at 57 N -> 60.5 km degree metric OVERSTATES
Inverness (57.5 N) to Portree (57.4 N), almost due west:
true, on the grid 118 km
degree metric 132 km +11.4%
+--------------+-----------+-----------+-------+
| pair | grid km | degree km | ratio |
+--------------+-----------+-----------+-------+
| Inverness - | 118 | 132 | 1.114 |
| Portree | | | |
+--------------+-----------+-----------+-------+
The lesson is not "never approximate". It is "know where your
approximation fails, and check whether your data lives there".
Editor8HOLOS8Fuseki8bookshop-trail-full.ttl
Q56
Everything within fifty kilometres
Which shops are within 50 km of York, and how far is each?
A radius test is a comparison against a squared threshold: rather than rooting the distance, square the limit. 50 km is 50,000 metres, so the test is d2 < 2,500,000,000 -- and no square root is needed anywhere.
What to take away
Compare against a squared threshold rather than taking a square root. The answer is exact.
This makes radius queries available in any engine that can multiply.
Add a bounding box first when the dataset is large: it removes most candidates before the multiplication runs.
want: sqrt(dx^2 + dy^2) < 50000
but no sqrt available, so square both sides:
dx^2 + dy^2 < 50000^2
dx^2 + dy^2 < 2 500 000 000
+--------------------------------------+
| .---------. |
| / *York \ |
| | * Endpapers | |
| | * Bookwyrm | r = 50km |
| \ / |
| '---------' |
| o Whitby's shop (58 km) |
+--------------------------------------+
Squaring the threshold instead of rooting the distance is the
same trick as q53, used the other way round. It is exact, not
an approximation.
Editor2HOLOS2Fuseki2bookshop-trail-full.ttl
Module 10
GeoSPARQL proper
The same questions, asked with geof: functions against WKT geometries. Shorter, exact, and dependent on an engine that implements them. HOLOS and a GeoSPARQL-enabled Fuseki do; the browser editor doesn't.
# Q57 The same question, one function
# How far is each shop from Hay-on-Wye, in kilometres?
# Data: bookshop-trail-full.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX geo: <http://www.opengis.net/ont/geosparql#>
PREFIX geof: <http://www.opengis.net/def/function/geosparql/>
SELECT ?name (ROUND(?metres / 100) / 10 AS ?km)
WHERE {
bt:place-hay-on-wye geo:hasDefaultGeometry/geo:asWKT ?origin .
?shop a bs:Bookshop ;
rdfs:label ?name ;
geo:hasDefaultGeometry/geo:asWKT ?here .
BIND( geof:distance(?origin, ?here, <http://www.opengis.net/def/uom/OGC/1.0/metre>) AS ?metres )
}
ORDER BY ?metres
LIMIT 12
geof:distance takes two geometries and a unit and returns a real great-circle distance. Everything q53 approximated with scaling and squaring becomes a single function call -- provided the engine implements it.
What to take away
geof:distance replaces a page of arithmetic, and is exact.
GeoSPARQL puts the geometry on its own node: reach the literal with geo:hasDefaultGeometry/geo:asWKT.
The unit is an argument. Ask for metres and you get metres; there's no implicit default worth relying on.
q53, portable: q57, GeoSPARQL:
BIND((?lat - 52.076)*111.19 geof:distance(?g1, ?g2, uom:metre)
AS ?dy)
BIND((?long + 3.1288)*66.70
AS ?dx)
BIND(?dx*?dx + ?dy*?dy
AS ?d2)
ORDER BY ?d2 ORDER BY ?metres
squared kilometres, metres, exact,
approximate, on the ellipsoid
runs everywhere needs GeoSPARQL
The geometry lives on a separate node, which is why the path has
two steps:
?shop --geo:hasDefaultGeometry--> ?g --geo:asWKT--> "POINT(...)"
Note. HOLOS only, and for a precise reason. Comunica has no geof: functions at all. Jena does have them -- run scripts/setup-geosparql.ps1 and the whole library appears -- but in this 6.2.0 build every function that returns a LINEAR measure comes back unbound: geof:distance in metres or kilometres, geof:area, geof:length. No error, no warning, HTTP 200, just an empty column. geof:distance in degrees or radians does work, as do all the topological functions and all the geometry constructors. So q58 runs on Fuseki and this one doesn't. Module 09 answers the same question with arithmetic, on every engine.
EditorHOLOS12Fusekibookshop-trail-full.ttl
Q58
Which area is this point inside
Verify that every settlement really does fall inside the polygon of the council area it claims to be in.
# Q58 Which area is this point inside
# Verify that every settlement really does fall inside the polygon of the council area it claims to be in.
# Data: bookshop-trail-full.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX geo: <http://www.opengis.net/ont/geosparql#>
PREFIX geof: <http://www.opengis.net/def/function/geosparql/>
SELECT ?settlement ?area
WHERE {
?s a bs:Settlement ;
rdfs:label ?settlement ;
bs:within ?c ;
geo:hasDefaultGeometry/geo:asWKT ?point .
?c a bs:CouncilArea ;
rdfs:label ?area ;
geo:hasDefaultGeometry/geo:asWKT ?polygon .
FILTER( LANG(?settlement) = "en" )
FILTER( !geof:sfWithin(?point, ?polygon) )
}
geof:sfWithin tests one geometry against another using the Simple Features relation. Here it checks the dataset against itself: the bs:within link says a town is in a council area, and the geometry should agree. Zero rows means the data is consistent.
What to take away
Topological relations are functions returning a boolean, usable directly in a FILTER.
A query that should return zero rows is a test. Keep it and run it whenever the data changes.
sfWithin, sfContains and sfIntersects cover most needs; the Egenhofer and RCC8 families are there when you need finer distinctions.
two independent statements about the same fact:
symbolic: bt:place-york bs:within bt:place-north-yorkshire
geometric: POINT(-1.0873 53.96) inside POLYGON((...))
this query asks whether they ever disagree:
?s bs:within ?c
FILTER( !geof:sfWithin(?pointOfS, ?polygonOfC) )
^
+-- NOT within -> a contradiction
result: 0 rows. The polygons were computed from the settlements
they contain, so containment is true by construction -- and this
query is how you prove it rather than assert it.
The Simple Features family: sfWithin, sfContains, sfIntersects,
sfOverlaps, sfTouches, sfCrosses, sfDisjoint, sfEquals.
Note. Returns zero rows when the data is sound, which is the point. Runs on Fuseki as well as HOLOS once scripts/setup-geosparql.ps1 has been run: Jena's GeoSPARQL handles the topological functions (sfWithin, sfIntersects, sfContains and the rest) perfectly well. Check it with the positive form -- FILTER(geof:sfWithin(...)) without the negation -- which finds all 30 settlements.
EditorHOLOS0Fuseki0bookshop-trail-full.ttl
Q59
How long is the trail
Measure each trail segment from its geometry, and compare with the distance recorded in the data.
# Q59 How long is the trail
# Measure each trail segment from its geometry, and compare with the distance recorded in the data.
# Data: bookshop-trail-full.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX geo: <http://www.opengis.net/ont/geosparql#>
PREFIX geof: <http://www.opengis.net/def/function/geosparql/>
SELECT ?label ?storedKm ?measuredKm
WHERE {
?seg a bs:TrailSegment ;
rdfs:label ?label ;
bs:distanceKm ?storedKm ;
geo:hasDefaultGeometry/geo:asWKT ?line .
BIND( ROUND(geof:length(?line) / 100) / 10 AS ?measuredKm )
}
ORDER BY DESC(?storedKm)
LIMIT 10
geof:length measures a LineString. The dataset also stores bs:distanceKm, computed when the data was built as the straight-line distance between the endpoints. The segment geometry bends through a midpoint, so the measured length should be a little longer -- and if it weren't, something would be wrong.
What to take away
geof:length, geof:area and geof:perimeter measure geometries; geof:envelope, geof:convexHull, geof:buffer and geof:boundary derive new ones.
Storing a derived number and measuring it are different things. Comparing them catches errors.
Deriving geometry in the query rather than storing it keeps the data smaller, at the cost of doing the work every time.
the geometry is a three-point line, deliberately bent:
shop A *---------.
* midpoint, nudged sideways
.--------'
shop B *-'
bs:distanceKm = straight line A to B (stored)
geof:length = along the bent line (measured)
measured > stored, always, by the amount of the bend.
Two numbers that should differ in a known direction are a good
consistency check: if the sign ever flips, a geometry has been
written wrongly.
Note. geof:length is a linear measure, so it shares q57's fate on Jena: registered, callable, and unbound on return. HOLOS answers it.
EditorHOLOS10Fusekibookshop-trail-full.ttl
Q60
Two coordinate systems, one query
Confirm that the National Grid geometry and the WGS84 geometry describe the same place.
# Q60 Two coordinate systems, one query
# Confirm that the National Grid geometry and the WGS84 geometry describe the same place.
# Data: bookshop-trail-full.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX geo: <http://www.opengis.net/ont/geosparql#>
PREFIX geof: <http://www.opengis.net/def/function/geosparql/>
SELECT ?name (ROUND(?separation) AS ?metresApart)
WHERE {
?place a bs:Settlement ;
rdfs:label ?name ;
geo:hasDefaultGeometry/geo:asWKT ?wgs84 .
?place geo:hasGeometry ?bngGeom .
?bngGeom geo:asWKT ?bng .
# !sameTerm, not !=. Two geo:wktLiteral values are not a datatype SPARQL
# knows how to compare, so `?wgs84 != ?bng` errors and the FILTER drops
# every row -- the q07 trap once more, this time on a geometry.
FILTER( !sameTerm(?wgs84, ?bng) )
FILTER( LANG(?name) = "en" )
BIND( geof:distance(?wgs84, ?bng,
<http://www.opengis.net/def/uom/OGC/1.0/metre>) AS ?separation )
}
ORDER BY DESC(?separation)
LIMIT 10
PREFIXbs:<https://example.org/bookshop-trail/schema#>PREFIXrdfs:<http://www.w3.org/2000/01/rdf-schema#>PREFIXgeo:<http://www.opengis.net/ont/geosparql#>PREFIXgeof:<http://www.opengis.net/def/function/geosparql/>SELECT?name (ROUND(?separation) AS?metresApart)
WHERE {
?placeabs:Settlement ;
rdfs:label?name ;
geo:hasDefaultGeometry/geo:asWKT?wgs84 .
?placegeo:hasGeometry?bngGeom .
?bngGeomgeo:asWKT?bng .
# !sameTerm, not !=. Two geo:wktLiteral values are not a datatype SPARQL# knows how to compare, so `?wgs84 != ?bng` errors and the FILTER drops# every row -- the q07 trap once more, this time on a geometry.FILTER( !sameTerm(?wgs84, ?bng) )
FILTER( LANG(?name) = "en" )
BIND( geof:distance(?wgs84, ?bng,
<http://www.opengis.net/def/uom/OGC/1.0/metre>) AS?separation )
}
ORDERBYDESC(?separation)
LIMIT10
How it works
Each settlement carries two geometries: a CRS84 point in degrees and an EPSG:27700 point in metres. An engine that reads the CRS URI at the front of a WKT literal will convert before comparing, and find the distance between them is essentially zero. An engine that assumes everything is CRS84 will compute a nonsensical answer, because it will read 425000 as a longitude.
What to take away
The coordinate reference system is part of the WKT literal's value.
Mixing systems in one dataset is normal; whether your engine copes is the question to ask before you rely on it.
An engine that silently assumes CRS84 won't error. It will give you a wrong number, which is worse.
Use !sameTerm to say 'a different term'. Inequality on two literals of a datatype the engine can't order is an error, and an error in a FILTER removes the row -- so the obvious spelling of this query returns nothing at all.
the CRS is part of the literal, not metadata about it:
"<...CRS84> POINT(-1.0873 53.96)"^^geo:wktLiteral
-----+---- --+--- --+--
| | +-- latitude, degrees
| +---------- longitude, degrees
+-- the reference system
"<...EPSG/0/27700> POINT(460000.0 452000.0)"^^geo:wktLiteral
--------+------- ---+--- ---+---
| | +-- northing, metres
| +----------- easting, metres
+-- a different system entirely
geof:distance between them, correctly handled: ~0 metres
the same, if 460000 is read as a longitude: nonsense
HOLOS reads CRS84, EPSG:4326, EPSG:27700 and EPSG:3857, and
refuses a system it does not know rather than guessing. Check
what your own engine does before you trust a mixed dataset.
Note. HOLOS-specific in practice: it's the engine among the three that reads EPSG:27700. The lesson -- that a CRS URI is part of the value and must be honoured -- is general.
EditorHOLOS10Fusekibookshop-trail-full.ttl
Module 11
SPARQL 1.2 and RDF 1.2
How to say something about a statement, and how to ask about it afterwards. Triple terms, the annotation syntax, and language strings that know which way they are written.
# Q61 Who says the shop opened when
# The founding dates are disputed. Who claims what, and how much do we trust them?
# Data: bookshop-trail-1.2.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?shopName ?year ?sourceName ?confidence
WHERE {
?shop bs:founded ?year {| bs:claimedBy ?source ; bs:confidence ?confidence |} .
?shop rdfs:label ?shopName .
?source rdfs:label ?sourceName .
}
ORDER BY ?shopName DESC(?confidence)
RDF 1.2 lets a statement be annotated with further statements about it. In the query, {| ... |} after a triple pattern matches those annotations, binding the variables inside. The base triple and its annotation come back in one row, with no intermediate node to invent or to remember.
What to take away
{| ... |} in a query matches annotations on the triple pattern it follows.
A dataset can hold two contradictory claims without being broken, as long as each is attributed.
The syntax is symmetrical: the same {| |} that writes an annotation in Turtle reads it in SPARQL.
in the data:
bt:shop-ex-libris bs:founded 1919
{| bs:claimedBy bt:source-national-register ;
bs:confidence 0.99 |} .
bt:shop-ex-libris bs:founded 1921
{| bs:claimedBy bt:source-local-paper ;
bs:confidence 0.40 |} .
in the query, the same shape:
?shop bs:founded ?year {| bs:claimedBy ?src ; bs:confidence ?c |} .
--------+------- -----------------+-------------------
the statement what is said ABOUT the statement
+-------------+------+-------------------+------+
| Ex Libris | 1919 | national-register | 0.99 |
| Ex Libris | 1921 | local paper | 0.40 |
+-------------+------+-------------------+------+
Both claims are in the graph. Neither is privileged. Deciding
between them is the query's job, not the data's -- see q63.
Editor10HOLOS10Fuseki10bookshop-trail-1.2.ttl
Q62
Find the contradictions
Which shops have two different founding dates on record?
Join the annotated pattern to itself and keep the pairs where the years differ. The self-join is on the shop; the inequality does the rest. The STR comparison on the sources keeps each disagreeing pair once rather than twice.
What to take away
An annotated pattern joins to itself exactly as a plain one does.
Comparing the source IRIs as strings is the standard way to emit an unordered pair once instead of twice.
Recording disagreement is more useful than resolving it at load time, because the right resolution depends on the question.
STR() around both years, and for the same reason as q07: comparing two xsd:gYear values directly returns nothing at all on two of the three engines. Written the obvious way, this query reports no contradictions and looks like good news.
?shop bs:founded ?yearA {| bs:claimedBy ?srcA |} .
?shop bs:founded ?yearB {| bs:claimedBy ?srcB |} .
^ ^
+-- same shop ---+ different years
FILTER( STR(?yearA) != STR(?yearB) ) <- a genuine contradiction
FILTER( STR(?srcA) < STR(?srcB) ) <- report each pair once
Ex Libris 1919 vs 1921
Endpapers 1949 vs 1946
Candlemas 1931 vs 1928
Castle Steps 1962 vs 1965
Gutter and Gilt does NOT appear: two sources, same year, so
there is no contradiction -- there is corroboration.
Editor4HOLOS4Fuseki4bookshop-trail-1.2.ttl
Q63
Believe the best source
For each shop, which founding date has the strongest backing?
A sub-query finds the highest confidence attached to any founding claim about each shop; the outer query re-joins to recover the year that claim asserted. It's exactly the 'group, then join back' shape from q38, applied to annotations rather than to events.
What to take away
Annotations turn 'which fact is true?' from a data-modelling problem into a query.
Keep the conflicting claims and resolve them at query time; different questions deserve different resolutions.
The aggregate-then-rejoin pattern works on annotation values exactly as it does on ordinary ones.
step 1 -- best confidence per shop
SELECT ?shop (MAX(?c) AS ?best)
WHERE { ?shop bs:founded ?y {| bs:confidence ?c |} }
GROUP BY ?shop
|
step 2 -- which claim had it?
?shop bs:founded ?year {| bs:confidence ?best |} .
----+
the join condition -+
+--------------+------+------+-------------------+
| Ex Libris | 1919 | 0.99 | national register |
| Endpapers | 1949 | 0.97 | national register |
| Candlemas | 1931 | 0.98 | national register |
| Castle Steps | 1962 | 0.85 | trail guide 2024 |
+--------------+------+------+-------------------+
The graph keeps every claim. The query chooses. Change the
policy -- most recent, most sources, highest confidence -- and
only the query changes.
Editor5HOLOS5Fuseki5bookshop-trail-1.2.ttl
Q134
Naming a statement in the query itself
Find the shop whose founding year somebody disputes, using the reifier syntax rather than a variable.
# Q134 Naming a statement in the query itself
# Find the shop whose founding year somebody disputes, using the reifier syntax rather than a variable.
# Data: bookshop-trail-1.2.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
SELECT ?shop ?year ?confidence
WHERE {
<< ?shop bs:founded ?year ~ ?r >> .
?r bs:confidence ?confidence .
}
ORDER BY ?confidence ?shop
<< s p o ~ ?r >> is the reifier form: it matches the triple and binds ?r to the reifier that identifies it, in one piece of syntax. It is the query-side counterpart of the annotation syntax the data is written in, and it saves the two-step of matching the triple term and then finding what rdf:reifies it.
What to take away
<< s p o ~ ?r >> matches a triple and binds its reifier in one step.
The reifier is what carries the annotations. The triple term is the statement; the reifier is this particular assertion of it.
Reifiers are usually blank nodes, so do not sort or page on one -- module 16 is about why.
the data, written with annotation syntax:
bt:shop-ex-libris bs:founded "1919"^^xsd:gYear
{| bs:statedOn bt:source-guide ; bs:confidence 0.7 |} .
what that means underneath:
_:r rdf:reifies <<( bt:shop-ex-libris bs:founded "1919" )>> .
_:r bs:statedOn bt:source-guide .
two ways to get at it:
the long way the reifier syntax
------------ ------------------
?r rdf:reifies ?t . << ?s bs:founded ?y ~ ?r >>
BIND(SUBJECT(?t) AS ?s) ?r bs:confidence ?c .
BIND(OBJECT(?t) AS ?y)
?r bs:confidence ?c .
Same answer, and the second says what it means. The ~ ?r part
is optional: << ?s ?p ?o >> on its own matches the triple
through some reifier without naming it.
+----------------------------------------------------------+
| A reifier identifies a STATEMENT, not a fact. Two |
| reifiers on the same triple are two separate claims |
| about it -- one from a guidebook, one from the shop -- |
| and that is the whole point of the mechanism. |
+----------------------------------------------------------+
Editor10HOLOS10Fuseki10bookshop-trail-1.2.ttl
Q64
What the annotation syntax really is
Strip away the sugar: what triples does {| ... |} actually create?
# Q64 What the annotation syntax really is
# Strip away the sugar: what triples does {| ... |} actually create?
# Data: bookshop-trail-1.2.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
SELECT ?statement ?p ?o
WHERE {
?reifier rdf:reifies ?statement ;
?p ?o .
FILTER( ?p != rdf:reifies )
# Narrow to the two rival claims about one shop, so the expansion is
# small enough to read. Ordering by ?reifier would not work: reifiers
# are blank nodes, and no two engines sort those alike.
FILTER( SUBJECT(?statement) = bt:shop-ex-libris )
FILTER( PREDICATE(?statement) = bs:founded )
}
ORDER BY ?p ?o
PREFIXbt:<https://example.org/bookshop-trail/>PREFIXbs:<https://example.org/bookshop-trail/schema#>PREFIXrdf:<http://www.w3.org/1999/02/22-rdf-syntax-ns#>SELECT?statement?p?oWHERE {
?reifierrdf:reifies?statement ;
?p?o .
FILTER( ?p != rdf:reifies )
# Narrow to the two rival claims about one shop, so the expansion is# small enough to read. Ordering by ?reifier would not work: reifiers# are blank nodes, and no two engines sort those alike.FILTER( SUBJECT(?statement) = bt:shop-ex-libris )
FILTER( PREDICATE(?statement) = bs:founded )
}
ORDERBY?p?o
How it works
An annotation is shorthand. Writing `s p o {| a b |}` asserts the base triple, mints a reifier, links it to a triple term with rdf:reifies, and hangs the annotation off the reifier. This query asks for the reifier directly, which is what the shorthand was hiding.
What to take away
{| ... |} is syntactic sugar over rdf:reifies plus a triple term.
A triple term <<( s p o )>> is one RDF term, legal only in object position. This is the main thing RDF 1.2 changed from the earlier RDF-star drafts.
The base triple is still asserted. Annotating a statement doesn't make it hypothetical.
The reifier is usually a blank node, so never sort or page on it. This query filters on the statement's subject and predicate instead, using the SPARQL 1.2 term functions from q66.
what you write:
bt:shop-ex-libris bs:founded 1919 {| bs:confidence 0.99 |} .
what the parser produces -- four things, three of them triples:
bt:shop-ex-libris bs:founded 1919 . <- the base triple
_:r rdf:reifies <<( bt:shop-ex-libris bs:founded 1919 )>> .
_:r bs:confidence 0.99 .
^ ^
| +-- an ordinary triple about _:r
+-- _:r is the "reifier": a name for the statement
<<( s p o )>> is a TRIPLE TERM: a single RDF term whose
value is a triple. It may only appear as an object.
So the annotation syntax is not a new kind of data. It is
ordinary triples, with a term type for talking about statements.
Editor4HOLOS4Fuseki4bookshop-trail-1.2.ttl
Q65
A statement as the object of a statement
Which claims does the National Register explicitly reject?
# Q65 A statement as the object of a statement
# Which claims does the National Register explicitly reject?
# Data: bookshop-trail-1.2.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?sourceName ?shopName ?disputedYear
WHERE {
?source bs:disputes <<( ?shop bs:founded ?disputedYear )>> .
?source rdfs:label ?sourceName .
?shop rdfs:label ?shopName .
}
ORDER BY ?shopName
A triple term can be the object of an ordinary triple, which is how you point at a statement without asserting it. bs:disputes does exactly that: the disputed statement appears inside <<( ... )>> and isn't thereby claimed to be true.
What to take away
A triple term names a statement without asserting it -- the thing reification was always trying to do.
Triple terms may appear in a query pattern with variables inside, and those variables bind.
Use annotation syntax when the statement is true and you want to say more about it; use a bare triple term when it may not be.
bt:source-national-register bs:disputes
<<( bt:shop-ex-libris bs:founded "1921"^^xsd:gYear )>> .
+---------------------------------------------------+
| the OBJECT of this triple is itself a triple |
| |
| and crucially, it is NOT asserted: |
| saying "X disputes S" does not put S in the graph |
+---------------------------------------------------+
contrast with the annotation syntax, which DOES assert:
s p o {| ... |} asserts s p o
?x bs:disputes <<(s p o)>> does not
matching one in a query, with variables inside:
?source bs:disputes <<( ?shop ?prop ?value )>>
binds ?shop, ?prop and ?value from inside the triple term.
Editor3HOLOS3Fuseki3bookshop-trail-1.2.ttl
Q66
Taking a triple term apart
Pull the subject, predicate and object out of every disputed statement.
SPARQL 1.2 adds functions over triple terms: isTRIPLE tests for one, and SUBJECT, PREDICATE and OBJECT take it apart. Together they let a query reason about statements it did not know the shape of in advance.
What to take away
isTRIPLE, SUBJECT, PREDICATE, OBJECT and TRIPLE are the SPARQL 1.2 term functions.
They make generic, shape-agnostic queries over statements possible.
Test with isTRIPLE before decomposing: SUBJECT of a non-triple is an error, and an error in a FILTER quietly drops the row.
?t = <<( bt:shop-ex-libris bs:founded "1921"^^xsd:gYear )>>
isTRIPLE(?t) --> true
SUBJECT(?t) --> bt:shop-ex-libris
PREDICATE(?t) --> bs:founded
OBJECT(?t) --> "1921"^^xsd:gYear
and in the other direction:
TRIPLE(?s, ?p, ?o) --> a new triple term
+----------------------------------------------+
| these work on ANY triple term, whatever its |
| shape -- so a query can inspect statements |
| whose predicate it does not know |
+----------------------------------------------+
Verified on all three engines: isTRIPLE, SUBJECT, PREDICATE,
OBJECT and TRIPLE all evaluate. VERSION() does not -- Jena has
it, the other two do not.
# Q67 Text that knows which way it runs
# Which labels are written right to left?
# Data: bookshop-trail-1.2.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?label ?language ?direction
WHERE {
?thing rdfs:label ?label .
FILTER( isLITERAL(?label) && hasLANGDIR(?label) )
BIND( LANG(?label) AS ?language )
BIND( LANGDIR(?label) AS ?direction )
}
ORDER BY ?language ?label
RDF 1.2 adds a base direction to language-tagged strings, written @ar--rtl. LANGDIR returns it, hasLANGDIR tests for it, and STRLANGDIR constructs one. Without the direction, a renderer has to guess where to put a trailing bracket -- and it guesses wrong often enough to matter.
What to take away
@lang--dir attaches a base direction to a literal; LANGDIR reads it.
hasLANGDIR distinguishes a directional literal from a plain language-tagged one, which matters because most data has neither.
Direction is a rendering fact, not a linguistic one, and RDF 1.1 had nowhere to put it.
"البحر المظلم"@ar--rtl
------+----- -+- -+-
| | +-- base direction: rtl
| +------ language: Arabic
+--------------- the text
LANGDIR(?l) --> "rtl"
hasLANGDIR(?l) --> true (false for plain @ar)
LANG(?l) --> "ar" (unchanged from 1.1)
STRLANGDIR("hi","en","ltr") --> "hi"@en--ltr
why it matters:
title (2019) with direction, the bracket is placed
(2019) title without it, the renderer may guess wrongly
A language tag says how to pronounce it. A direction says how
to lay it out. They are different questions.
Editor3HOLOS3Fuseki3bookshop-trail-1.2.ttl
Q138
Building and matching language tags
Which labels carry a language tag, which of them are English, and how do you add a tag to a string that has none?
# Q138 Building and matching language tags
# Which labels carry a language tag, which of them are English, and how do you add a tag to a string that has none?
# Data: bookshop-trail-1.2.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?tagged ?english (COUNT(*) AS ?labels)
WHERE {
?thing rdfs:label ?label .
BIND( IF( hasLANG(?label), "tagged", "no tag" ) AS ?tagged )
BIND( IF( hasLANG(?label) && langMatches( LANG(?label), "en" ),
"English", "other" ) AS ?english )
# The repair, on anything that arrived without a tag.
BIND( IF( hasLANG(?label), ?label,
STRLANG( STR(?label), "en" ) ) AS ?repaired )
BIND( STRLANGDIR( STR(?label), "ar", "rtl" ) AS ?asArabic )
}
GROUP BY ?tagged ?english
ORDER BY ?tagged ?english
PREFIXrdfs:<http://www.w3.org/2000/01/rdf-schema#>SELECT?tagged?english (COUNT(*) AS?labels)
WHERE {
?thingrdfs:label?label .
BIND( IF( hasLANG(?label), "tagged", "no tag" ) AS?tagged )
BIND( IF( hasLANG(?label) && langMatches( LANG(?label), "en" ),
"English", "other" ) AS?english )
# The repair, on anything that arrived without a tag.BIND( IF( hasLANG(?label), ?label,
STRLANG( STR(?label), "en" ) ) AS?repaired )
BIND( STRLANGDIR( STR(?label), "ar", "rtl" ) AS?asArabic )
}
GROUPBY?tagged?englishORDERBY?tagged?english
How it works
Four functions round out the language handling. hasLANG asks whether a literal is tagged at all; langMatches compares a tag against a range, which is what makes en-GB an English label; STRLANG builds a tagged literal from a plain string, and STRLANGDIR builds one that also carries a direction.
What to take away
langMatches compares a language tag against a range, so "en" matches "en-GB". Equality on LANG() does not, and loses data silently.
hasLANG tests whether a literal is tagged at all -- the SPARQL 1.2 way of writing LANG(?x) != "".
STRLANG and STRLANGDIR build tagged literals from plain strings, which is how untagged text gets repaired.
hasLANG("Hay-on-Wye"@en) true
hasLANG("Hay-on-Wye") false
hasLANG(220) false
langMatches( LANG(?x), "en" ) the RIGHT test
"en" matches "en-GB" matches
"en-US" matches "cy" does not
LANG(?x) = "en" the test people write
"en" matches "en-GB" DOES NOT
+----------------------------------------------------------+
| A language range is a prefix match on subtags, not a |
| string comparison. Data that arrives with regional tags |
| disappears from every query written the second way, and |
| it disappears silently. |
+----------------------------------------------------------+
going the other way, building a literal:
STRLANG("Hay-on-Wye", "en") "Hay-on-Wye"@en
STRLANGDIR("البحر المظلم", "ar", "rtl") ...@ar--rtl
STRLANG is how untagged text gets repaired on the way in.
STRLANGDIR is its RDF 1.2 counterpart, and the direction is
what tells a renderer which way to lay the text out -- q67 is
where that matters.
"*" as a range matches anything tagged at all, which makes
langMatches(LANG(?x), "*") a wordier hasLANG(?x).
measured on this dataset:
tagged, English 410 labels
tagged, other 34 labels
no tag 0
The repair branch never fires, because every label here was
written with a tag. That is what a clean dataset looks like,
and it is worth knowing that yours is unusual if it matches.
Editor2HOLOS2Fuseki2bookshop-trail-1.2.ttl
Q68
The same fact, modelled twice
Stock levels are in this dataset twice over -- once the RDF 1.1 way and once the 1.2 way. Compare them.
# Q68 The same fact, modelled twice
# Stock levels are in this dataset twice over -- once the RDF 1.1 way and once the 1.2 way. Compare them.
# Data: bookshop-trail-1.2.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?shopName ?title ?copiesVia11 ?copiesVia12
WHERE {
?record a bs:StockRecord ;
bs:atShop ?shop ;
bs:ofWork ?work ;
bs:copies ?copiesVia11 .
?shop bs:stocks ?work {| bs:copies ?copiesVia12 |} .
?shop rdfs:label ?shopName .
?work rdfs:label ?title .
}
ORDER BY ?shopName ?title
The 1.1 model invents a bs:StockRecord node to carry copies and price. The 1.2 model annotates the bs:stocks link directly. Both answer the question; the query shows what each costs to write, and confirms they agree.
What to take away
The n-ary relation pattern -- invent a node -- is how RDF 1.1 says anything about a relationship, and it works.
RDF 1.2 annotations remove the invented node, keeping the relationship queryable as a plain triple.
Model with an intermediate node when the relationship is a thing in its own right; annotate when it's only a link you want to qualify.
RDF 1.1 -- invent a node RDF 1.2 -- annotate the link
bt:stock-inkwell--the-book-town bt:shop-inkwell
a bs:StockRecord ; bs:stocks bt:book-the-book-town
bs:atShop bt:shop-inkwell ; {| bs:copies 12 ;
bs:ofWork bt:book-... ; bs:shelfPrice 9.99 |} .
bs:copies 12 ;
bs:shelfPrice 9.99 .
5 triples, 1 invented IRI 3 triples, 1 blank reifier
the link is implicit the link is a real triple
every query goes via the record simple queries stay simple
querying them:
?r bs:atShop ?shop ; ?shop bs:stocks ?work
bs:ofWork ?work ; {| bs:copies ?n |} .
bs:copies ?n .
Neither is wrong. The 1.1 form is better when the relationship
has its own identity and lifecycle; the 1.2 form is better when
you just want to say a bit more about a link that already exists.
Editor24HOLOS24Fuseki24bookshop-trail-1.2.ttl
Module 12
Graphs in, graphs out
Module 07 introduced ASK, CONSTRUCT and DESCRIBE. This one uses them in earnest. The shift is in what SPARQL is for: not answering a question and printing a table, but testing a condition, or taking a graph in and handing a different graph back. Every query here returns a boolean or RDF -- not one of them returns a table.
# Q75 Can you walk from Wigtown to London
# Is there any route along the trail from The Inkwell to Ex Libris?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
ASK {
bt:shop-inkwell (bs:connectsTo|^bs:connectsTo)+ bt:shop-ex-libris .
}
A property path inside an ASK. The engine needs to find one route, not all of them, and may stop the moment it does. That makes ASK the right form for a reachability question whose answer you're going to act on rather than read.
What to take away
ASK returns one boolean and permits the engine to stop at the first solution.
A positive existence question is cheap. Its negation isn't, because nothing can be concluded until the search is exhausted.
Reach for ASK when a program will branch on the answer; reach for SELECT when a person will read it.
ASK {
bt:shop-inkwell (bs:connectsTo|^bs:connectsTo)+ bt:shop-ex-libris .
}
|
+-------+--------+
v v
true false
a route exists no route
--> true
The same path in a SELECT returns 31 shops and has to find them
all. Here the engine may stop at the first success, because one
is all the question needs.
Compare with the negative form, which cannot stop early:
ASK { FILTER NOT EXISTS { ...path... } }
To prove a route does NOT exist, every possibility must be
eliminated. "Is there one?" is cheap; "is there none?" is not.
# Q76 The assertion that must come back false
# Is any bookshop missing a label?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
ASK {
?shop a bs:Bookshop .
FILTER NOT EXISTS { ?shop rdfs:label ?label }
}
An ASK written so that `false` is the healthy answer. Phrased this way it's a test rather than a question, and it can be run automatically: load the data, ask, and fail the build if the answer is true.
What to take away
An ASK whose expected answer is false is a test, and belongs in whatever runs your builds.
ASK tells you that something is wrong; SHACL tells you what and where. They answer different questions and both are cheap.
Writing the assertion negatively -- 'is anything broken?' -- keeps the healthy answer constant as the dataset grows.
ASK {
?shop a bs:Bookshop .
FILTER NOT EXISTS { ?shop rdfs:label ?label }
}
--> false the data is sound
A suite of these is a cheap integrity check, and it needs no
SHACL processor:
any shop with no town? false ok
any work with no author? false ok
any segment joining a shop false ok
to itself?
any place inside itself? false ok
Turn one true and you have found a bug. data/shapes.ttl says
the same things in SHACL, which reports WHICH node failed;
an ASK only reports THAT one did. Use ASK in a script, SHACL
when you need to fix what it finds.
Editor1HOLOS1Fuseki1bookshop-trail-1.1.ttl
Q77
One boolean per row, not one per query
For every shop, does it stock any translated fiction?
# Q77 One boolean per row, not one per query
# For every shop, does it stock any translated fiction?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?name ?stocksTranslated
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name .
BIND( EXISTS { ?shop bs:stocks/bs:genre bt:genre-translated-fiction }
AS ?stocksTranslated )
}
ORDER BY DESC(?stocksTranslated) ?name
ASK answers once for the whole query, which is no use when the question is really about each row. EXISTS is the row-wise form of the same test: it evaluates a pattern against the current bindings and yields a boolean you can bind, filter or select.
What to take away
EXISTS is the per-row version of ASK, and it can be bound to a variable rather than only used to filter.
BIND(EXISTS{...} AS ?flag) keeps every row and labels it. FILTER EXISTS removes rows. Choose by whether the absence is worth reporting.
The inner pattern sees the outer bindings, which is what makes it a test about this row rather than about the dataset.
ASK { ... } one boolean, for the whole query
EXISTS { ... } one boolean, for the current row
BIND( EXISTS { ?shop bs:stocks/bs:genre bt:genre-translated-fiction }
AS ?stocksTranslated )
--+--- ^
| +-- ?shop is bound from
+-- evaluated once per row the surrounding row
+----------------------+-------------------+
| Verso and Recto | true |
| Turn the Page | true |
| The Inkwell | false |
+----------------------+-------------------+
FILTER EXISTS is the same test used to drop rows. BIND EXISTS
keeps every row and records the answer, which is usually what a
report wants.
Editor33HOLOS33Fuseki33bookshop-trail-1.1.ttl
Q78
Asking a question about a total
Does any single town have three or more bookshops?
# Q78 Asking a question about a total
# Does any single town have three or more bookshops?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
ASK {
{
SELECT ?town (COUNT(?shop) AS ?shops)
WHERE {
?shop a bs:Bookshop ;
bs:locatedIn ?town .
}
GROUP BY ?town
HAVING ( COUNT(?shop) >= 3 )
}
}
ASK takes a whole group pattern, so it can contain a sub-query with grouping and HAVING. The sub-query produces a row only for towns that pass, and ASK then reports whether any row survived.
What to take away
ASK accepts any group pattern, sub-queries included, so any question you can express as 'are there any rows?' can be asked.
The sub-query does the counting; ASK only reports whether anything survived HAVING.
This is the cheap way to check a threshold without reading a table to find out.
ASK {
{ SELECT ?town (COUNT(?shop) AS ?n)
WHERE { ?shop a bs:Bookshop ; bs:locatedIn ?town }
GROUP BY ?town
HAVING ( COUNT(?shop) >= 3 ) }
}
|
the sub-query yields zero rows
|
v
false
Seven towns have two shops -- Wigtown, Edinburgh, Glasgow,
Sedbergh, York, London and Hay-on-Wye. None has three. Change
the 3 to a 2 and the answer becomes true.
An aggregate cannot appear at the top level of an ASK, because
there is nothing to group. Put it in a sub-query and the
question becomes "did that produce anything?"
Editor1HOLOS1Fuseki1bookshop-trail-1.1.ttl
Q79
Asking inside one named graph
Does the stock graph say anything at all about The Sea Margin?
# Q79 Asking inside one named graph
# Does the stock graph say anything at all about The Sea Margin?
# Data: bookshop-trail.trig
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
ASK {
GRAPH bt:graph-stock {
bt:shop-sea-margin bs:stocks ?anything .
}
}
An ASK wrapped in GRAPH asks about one part of the dataset rather than the whole of it. That turns it into a question about where a fact is filed, which is exactly what you want before joining across graphs that may not both be loaded.
What to take away
GRAPH inside ASK scopes the question to one named graph.
A pair of ASKs is the quickest way to find out what an unfamiliar endpoint actually has loaded.
Remember module 08: a pattern outside GRAPH sees only the default graph, which in this TriG file is nearly empty.
ASK { GRAPH bt:graph-stock { bt:shop-sea-margin bs:stocks ?anything } }
+------+------+
only this graph
--> true
Useful before a bigger query:
ASK { GRAPH bt:graph-stock { ?s ?p ?o } } is it loaded?
ASK { GRAPH bt:graph-claims { ?s ?p ?o } } is the 1.2 layer here?
An endpoint you did not load yourself may hold any subset of
what you expect. Two ASKs tell you which, in less time than
reading the documentation.
DESCRIBE takes a WHERE clause. The pattern selects the resources, and the engine then describes each one. It's the quickest way to pull a subgraph out of an endpoint when you don't yet know what the resources look like.
What to take away
DESCRIBE accepts a WHERE clause, and describes every resource the pattern binds.
The descriptions merge into a single graph; there's no boundary between them in the result.
Good for grabbing a subgraph to look at. Still engine-defined, so still not for a pipeline -- q82 shows the replacement.
DESCRIBE ?shop
WHERE {
?shop a bs:Bookshop ;
bs:locatedIn/bs:within+ bt:place-wales .
}
step 1 the WHERE clause finds 4 shops
step 2 the engine describes each of them
step 3 the descriptions are merged into ONE graph
+--------------+
| Cliff Road |--+
+--------------+ |
| Taff Margin |--+ one graph,
+--------------+ +-> not four
| Castle Steps |--+
+--------------+ |
| Clock Tower |--+
+--------------+
The result is a graph, so the shops are not separable in it
afterwards except by querying it again.
Editor74HOLOS74Fuseki74bookshop-trail-1.1.ttl
Q81
Describe several things at once
Describe a shop, the town it's in, and an author who lives there.
DESCRIBE takes a list of IRIs with no WHERE clause at all. The three descriptions come back merged, which is convenient when you want the neighbourhood of a few known resources and don't care where one ends and the next begins.
What to take away
DESCRIBE takes a bare list of IRIs, which is the shortest useful query in SPARQL.
It describes each resource independently and merges the results.
Incoming statements are usually absent. If you need 'what points at this?', ask for it explicitly with ^ or a second pattern.
DESCRIBE bt:shop-inkwell bt:place-wigtown bt:author-rab-fingal
no WHERE clause: the resources are named directly
shop-inkwell --+
place-wigtown --+--> one merged graph
author-rab-fingal +
Note what is NOT here. Nothing links the three in the result
unless the data already linked them -- DESCRIBE does not invent
connections, and it does not follow them either.
bt:shop-inkwell bs:locatedIn bt:place-wigtown is in the graph
because the shop's own description contains it. The reverse
direction is not, unless your engine chooses to include
incoming statements. Most do not.
Editor42HOLOS42Fuseki42bookshop-trail-1.1.ttl
Q82
The CONSTRUCT that replaces DESCRIBE
Get a description of The Quire that every engine will produce identically -- and that's actually readable.
# Q82 The CONSTRUCT that replaces DESCRIBE
# Get a description of The Quire that every engine will produce identically -- and that's actually readable.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
CONSTRUCT {
bt:shop-quire ?p ?o .
?o rdfs:label ?oLabel .
}
WHERE {
bt:shop-quire ?p ?o .
OPTIONAL { ?o rdfs:label ?oLabel . }
}
DESCRIBE leaves the choice of what to include to the engine. Writing the same thing as CONSTRUCT pins it down, and lets you add what a bare description always lacks: the labels of the things it points at, so the result reads without a second query.
What to take away
Anything DESCRIBE does, CONSTRUCT does explicitly and identically on every engine.
Pulling in the labels of referenced resources is what makes a description legible; DESCRIBE won't do it for you.
OPTIONAL around the label is required, or objects without one take their whole row with them.
Glasgow has a label in English and another in Gaelic, so the row for bs:locatedIn is produced twice and the base triple with it. Q44 explains why that changes the triple count on one engine and not the other two.
DESCRIBE bt:shop-quire engine decides. Not reproducible.
CONSTRUCT { you decide. Reproducible.
bt:shop-quire ?p ?o .
?o rdfs:label ?oLabel . <- the useful addition
}
WHERE {
bt:shop-quire ?p ?o .
OPTIONAL { ?o rdfs:label ?oLabel }
}
without the labels: with them:
bs:locatedIn bs:locatedIn
bt:place-glasgow bt:place-glasgow
bt:place-glasgow rdfs:label
"Glasgow"@en
The OPTIONAL matters: ?o is often a literal or a geometry node
with no label, and without it those statements vanish from the
output along with the rest of the row.
Engines differ, and that's expected. 26 triples from the browser editor, 25 from HOLOS and Fuseki, for the reason set out in q44 -- the template is instantiated once per solution, and bt:place-glasgow has two labels, so `bt:shop-quire bs:locatedIn bt:place-glasgow` is built twice. Comunica returns the stream; the other two return the set. Load either into a graph and they are identical.
Editor26HOLOS25Fuseki25bookshop-trail-1.1.ttl
Q83
Following one hop further
Describe The Sea Margin including its geometry, which lives on a separate node.
# Q83 Following one hop further
# Describe The Sea Margin including its geometry, which lives on a separate node.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX geo: <http://www.opengis.net/ont/geosparql#>
CONSTRUCT {
?s ?p ?o .
}
WHERE {
# The shop's own statements...
{ VALUES ?s { bt:shop-sea-margin }
?s ?p ?o . }
UNION
# ...and, separately, the statements of the node its geometry points to.
{ bt:shop-sea-margin geo:hasDefaultGeometry ?s .
?s ?p ?o . }
}
PREFIXbt:<https://example.org/bookshop-trail/>PREFIXgeo:<http://www.opengis.net/ont/geosparql#>CONSTRUCT {
?s?p?o .
}
WHERE {
# The shop's own statements...
{ VALUES?s { bt:shop-sea-margin }
?s?p?o . }
UNION# ...and, separately, the statements of the node its geometry points to.
{ bt:shop-sea-margingeo:hasDefaultGeometry?s .
?s?p?o . }
}
How it works
GeoSPARQL puts the coordinates on a geometry node, so a one-hop description of a shop contains a pointer and no numbers. The two halves are gathered with UNION rather than by nesting one inside the other -- and the reason why is worth more than the query.
What to take away
A resource's useful description rarely stops at one hop: geometry, addresses and n-ary nodes all sit one further out.
Nesting two independent patterns in one group multiplies the solutions. UNION gathers them without a cross product.
Decide the depth deliberately. Unbounded following turns a description into a copy of the dataset.
a one-hop description stops at a pointer:
bt:shop-sea-margin
geo:hasDefaultGeometry bt:geom-shop-sea-margin <- a pointer
wgs84:lat 57.41...
the coordinates are on the node it points to:
bt:geom-shop-sea-margin
geo:asWKT "<...CRS84> POINT(-6.19 57.41)"
THE WRONG WAY -- nest them in one solution:
?shop ?p ?o .
OPTIONAL { ?shop geo:hasDefaultGeometry ?geom . ?geom ?gp ?go }
17 shop statements x 2 geometry statements = 34 solutions,
each firing a two-triple template. The distinct triples are
still only 20, but the engine built 68 of them to get there --
and on an engine that does not deduplicate its CONSTRUCT
stream, all 68 come back.
THE RIGHT WAY -- keep the halves independent:
{ VALUES ?s { bt:shop-sea-margin } ?s ?p ?o }
UNION
{ bt:shop-sea-margin geo:hasDefaultGeometry ?s . ?s ?p ?o }
17 + 3 solutions. 20 triples. No multiplication anywhere.
Two patterns that share no variable but sit in the same group
multiply. UNION concatenates instead. When you are gathering
unrelated facts about different subjects, that is the one you
want.
Editor20HOLOS20Fuseki20bookshop-trail-1.1.ttl
Q84
CONSTRUCT WHERE, the short form
Extract the shops with their names and founding years, unchanged.
# Q84 CONSTRUCT WHERE, the short form
# Extract the shops with their names and founding years, unchanged.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
CONSTRUCT WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name ;
bs:founded ?year .
}
When the template you want is exactly the pattern you matched, the template can be omitted. `CONSTRUCT WHERE { ... }` uses the pattern as its own template. It's the standard way to cut a subgraph out of a larger one without retyping it.
What to take away
CONSTRUCT WHERE { ... } reuses the pattern as the template, and is the idiomatic way to extract a subgraph unchanged.
It accepts only a basic graph pattern: no FILTER, OPTIONAL, UNION or sub-query.
The moment you need to reshape anything, write the template out in full.
the long way:
CONSTRUCT { ?shop a bs:Bookshop ; rdfs:label ?n ; bs:founded ?y }
WHERE { ?shop a bs:Bookshop ; rdfs:label ?n ; bs:founded ?y }
--------------- identical ---------------
the short way:
CONSTRUCT WHERE { ?shop a bs:Bookshop ; rdfs:label ?n ; bs:founded ?y }
99 triples out: three per shop.
Restrictions, and the reason they exist -- the pattern IS the
template, so it must be something a template could contain:
no FILTER, no OPTIONAL, no UNION, no sub-query
just a basic graph pattern
Need any of those? Write the template out. That is q85.
Editor99HOLOS99Fuseki99bookshop-trail-1.1.ttl
Q85
Republish it in someone else's vocabulary
Turn the shops into schema.org, so a search engine could read them.
A template may use any vocabulary, not just the one the data is in. This is the everyday use of CONSTRUCT: your data stays in the model that suits you, and a query publishes it in the model your consumer expects. The blank node in the template builds structure that does not exist in the source at all.
What to take away
A CONSTRUCT template can use any vocabulary. Storing in one model and publishing in another is a query, not a migration.
Blank nodes in a template are minted fresh for each solution, so structure can be invented that the source doesn't have.
This is how you serve schema.org, or DCAT, or anything else, from data you did not model that way.
source model published model
bs:Bookshop schema:BookStore
rdfs:label schema:name
bs:founded schema:foundingDate
bs:locatedIn -> place -> label schema:address -> [ a PostalAddress ;
addressLocality ]
wgs84:lat / long schema:latitude / longitude
the blank node:
schema:address [ a schema:PostalAddress ;
schema:addressLocality ?town ]
^
+-- a FRESH blank node per solution, invented
by the template; nothing like it exists in
the source data
Each solution gets its own. Two shops never share one, which
is what you want here and is worth knowing when it is not.
Editor264HOLOS264Fuseki264bookshop-trail-1.1.ttl
Q86
Upgrade RDF 1.1 data to RDF 1.2
Turn the 95 StockRecord nodes into RDF 1.2 annotations, automatically.
This is the migration module 11 argues for, written as one query. TRIPLE() builds a triple term from three variables; IRI() mints a stable reifier from the record's own name; and the template emits the base triple, the rdf:reifies link and the annotations. Run it against the 1.1 file and the output is the 1.2 file's stock section.
What to take away
TRIPLE(s, p, o) constructs a triple term in an expression, which is how you get one into a CONSTRUCT template -- templates can't call functions themselves.
IRI(CONCAT(...)) mints names from data. Deriving them from something stable makes the transformation repeatable.
A whole modelling migration can be one query. Run it, check the output, load it, drop the old nodes.
in (RDF 1.1, an invented node) out (RDF 1.2)
bt:stock-inkwell--the-book-town bt:shop-inkwell bs:stocks
a bs:StockRecord ; bt:book-the-book-town .
bs:atShop ?shop ;
bs:ofWork ?work ; bt:reifier-inkwell--the-book-town
bs:copies 12 ; rdf:reifies <<( ?shop bs:stocks
bs:shelfPrice 9.99 . ?work )>> ;
bs:copies 12 ;
bs:shelfPrice 9.99 .
the two functions that make it work:
BIND( TRIPLE(?shop, bs:stocks, ?work) AS ?statement )
------+----- builds a triple term
+-- SPARQL 1.2
BIND( IRI(CONCAT("...reifier-", ?suffix)) AS ?reifier )
-+- a stable name, so
+-- re-running gives the same IRIs the output is idempotent
A blank node would work too, and would produce a different
graph every run. For a migration you want the same one.
Note. Reads the RDF 1.1 file and writes RDF 1.2, so it needs a SPARQL 1.2 engine to run even though its input is 1.1. All three qualify.
Editor380HOLOS380Fuseki380bookshop-trail-1.1.ttl
Q87
A profile of the genre scheme
Build a graph that records, for each genre, how many shops specialise in it and how many works are filed under it.
# Q87 A profile of the genre scheme
# Build a graph that records, for each genre, how many shops specialise in it and how many works are filed under it.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
CONSTRUCT {
?genre skos:prefLabel ?label ;
bs:shopCount ?shops ;
bs:workCount ?works .
}
WHERE {
?genre skos:prefLabel ?label .
FILTER( LANG(?label) = "en" )
{
SELECT ?genre (COUNT(DISTINCT ?shop) AS ?shops)
WHERE { ?shop bs:specialises/skos:broader* ?genre . }
GROUP BY ?genre
}
{
SELECT ?genre (COUNT(DISTINCT ?work) AS ?works)
WHERE { ?work bs:genre/skos:broader* ?genre . }
GROUP BY ?genre
}
}
Aggregation happens in sub-queries; the template assembles the results. Because `skos:broader*` climbs the tree, a shop specialising in Tartan Noir counts towards Crime Fiction and Fiction as well -- so the output is a rolled-up profile, not a flat tally.
What to take away
CONSTRUCT plus aggregation produces derived graphs -- summaries you can store, publish or query again.
Rolling a count up a SKOS tree is one path expression, and it's almost always what the reader expects a category total to mean.
The output is small. Load it back into the editor and the whole profile fits in the graph view.
two sub-queries, one template:
+ shops per genre, rolled up the tree ------+
| ?shop bs:specialises/skos:broader* ?genre |
+-------------------+-----------------------+
|
+ works per genre, rolled up ---------------+
| ?work bs:genre/skos:broader* ?genre |
+-------------------+-----------------------+
v
CONSTRUCT { ?genre bs:shopCount ?shops ;
bs:workCount ?works ;
skos:prefLabel ?label }
the rolling up, in one branch:
Cosy Crime 2 shops -+
Tartan Noir 1 shop -+-> Crime Fiction 4 shops
Crime Fiction 1 shop -+ +------> Fiction 17
+--> Literature 33
A flat count would put 1 against Crime Fiction and lose the
other three. The * is what makes the number mean what a reader
will assume it means -- and Literature, the top concept, ends up
with all 33 shops, which is the correct answer to "how many
shops specialise in some kind of literature?"
Editor63HOLOS63Fuseki63bookshop-trail-1.1.ttl
Q88
A validation report, as RDF
Produce a graph listing everything questionable in the dataset.
# Q88 A validation report, as RDF
# Produce a graph listing everything questionable in the dataset.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
CONSTRUCT {
[] a bs:DataIssue ;
bs:about ?thing ;
bs:message ?message .
}
WHERE {
{
?thing a bs:Work .
FILTER NOT EXISTS { ?thing bs:isbn ?isbn }
BIND( "Work has no ISBN. Expected for titles published before 1970." AS ?message )
}
UNION
{
?thing a bs:Bookshop .
FILTER NOT EXISTS { ?thing bs:website ?site }
BIND( "Shop publishes no website." AS ?message )
}
UNION
{
?thing a bs:Bookshop .
FILTER NOT EXISTS {
bt:shop-inkwell (bs:connectsTo|^bs:connectsTo)+ ?thing .
}
BIND( "Shop is not reachable on foot from the start of the trail." AS ?message )
}
}
PREFIXbt:<https://example.org/bookshop-trail/>PREFIXbs:<https://example.org/bookshop-trail/schema#>CONSTRUCT {
[] abs:DataIssue ;
bs:about?thing ;
bs:message?message .
}
WHERE {
{
?thingabs:Work .
FILTERNOTEXISTS { ?thingbs:isbn?isbn }
BIND( "Work has no ISBN. Expected for titles published before 1970."AS?message )
}
UNION
{
?thingabs:Bookshop .
FILTERNOTEXISTS { ?thingbs:website?site }
BIND( "Shop publishes no website."AS?message )
}
UNION
{
?thingabs:Bookshop .
FILTERNOTEXISTS {
bt:shop-inkwell (bs:connectsTo|^bs:connectsTo)+ ?thing .
}
BIND( "Shop is not reachable on foot from the start of the trail."AS?message )
}
}
How it works
Three checks in a UNION, each binding a message, and a template that mints a fresh blank node per finding. The result is a report you can query, diff against last week's, or hand to another tool -- which a printed table isn't.
What to take away
A report that's RDF can be queried, diffed and stored. A report that's a printed table can only be read.
[] in a template mints a fresh blank node per solution -- the right choice when the finding has no identity of its own.
UNION with a bound message is how you run several unrelated checks in one pass.
+ works with no ISBN -----------+
| FILTER NOT EXISTS {?w bs:isbn}|-+
+ shops with no website --------+ |
| FILTER NOT EXISTS {?s website}|-+- UNION -> CONSTRUCT
+ shops off the trail ----------+ | |
| FILTER NOT EXISTS {path} |-+ |
+-------------------------------+ v
[] a bs:DataIssue ;
bs:about ?thing ;
bs:message ?message .
^
+-- a fresh blank node for every finding
+----------------------+--------------------------------+
| bt:book-cold-harbour | Work has no ISBN |
| bt:shop-marginalia | Shop publishes no website |
| bt:shop-west-quay | Not reachable on foot from the |
| | start of the trail |
+----------------------+--------------------------------+
19 findings, 57 triples -- three per finding:
11 works published before 1970, so no ISBN existed
6 shops that publish no website
2 the south-western pair, joined to each other and
to nothing else
Every one is expected in this dataset. A report of
known-acceptable findings is still worth generating: what you
watch is the diff against last time.
Nothing in the data says two shops are neighbours; it says only which town each is in. The relationship is implied by the join, and CONSTRUCT is what turns an implication into a triple you can then traverse with a property path like any other.
What to take away
A relationship implied by a shared value becomes a real edge the moment you CONSTRUCT it.
STR(?a) < STR(?b) is the standard guard for emitting an unordered pair once rather than twice, and it also stops a thing pairing with itself.
Materialising a frequently-walked shortcut is a legitimate optimisation. Just be sure you can rebuild it when the source changes.
what the data says:
shop-inkwell bs:locatedIn place-wigtown
shop-marginalia bs:locatedIn place-wigtown
what it implies, and does not state:
shop-inkwell bs:sameTownAs shop-marginalia
the join that finds it:
?a bs:locatedIn ?town .
?b bs:locatedIn ?town . <- same ?town: that IS the relationship
FILTER( STR(?a) < STR(?b) ) <- each pair once, not twice
7 pairs, one per two-shop town: Wigtown, Edinburgh, Glasgow,
Sedbergh, York, London and Hay-on-Wye. 14 triples, because the
template asserts the link in both directions.
Load the result alongside the source and bs:sameTownAs is now
an ordinary predicate -- paths, counts and all. This is how a
graph grows a shortcut it uses often.
Editor14HOLOS14Fuseki14bookshop-trail-1.1.ttl
Module 13
Planning and debugging
Why a query is slow, why it returns nothing, and why it returns far too much. All three engines will show you the plan they built; this module reads those plans, and collects the diagnostic queries worth reaching for before you start rewriting anything.
A query says what you want; the engine decides
how. All three will show you what they decided, and the three
answers are usefully different because they show different layers of the
same idea.
# the query used throughout this sectionSELECT?name?siteWHERE {
?shopabs:Bookshop ; rdfs:label?name ; bs:locatedIn?town .
?townbs:within+ bt:place-scotland .
OPTIONAL { ?shopbs:website?site }
FILTER( STRLEN(?name) > 8 )
}
Jena — the algebra, before and after
qparse --print=op shows your query as the algebra the
specification defines; --print=opt shows what Jena will
actually run. Neither executes anything, which makes Jena the best of
the three for learning what a query means.
HOLOS — the physical plan
--explain prints the operator tree with the join
algorithm and the join keys on every node. Where Jena shows what,
HOLOS shows how: which side of each join is built into a hash table
and which side probes it. --reorder builds cardinality
statistics first and orders each basic graph pattern by estimated
selectivity.
The browser editor — Comunica
engine.explain(query, ctx, 'physical') returns the
operators it ran and the actor that handled each. Comunica is built
out of actors that bid for work, so its plan names the implementation
rather than only the operation. The SPARQL panel doesn't surface it,
so this one is a Node exercise; scripts/engines.py has a
working harness to adapt.
JENA, as written -- the filter is where you put it
(project (?name ?site)
(filter (> (strlen ?name) 8)
(leftjoin
(sequence
(bgp (triple ?shop rdf:type bs:Bookshop)
(triple ?shop rdfs:label ?name)
(triple ?shop bs:locatedIn ?town))
(path ?town (path+ bs:within) bt:place-scotland))
(bgp (triple ?shop bs:website ?site)))))
JENA, optimised -- the filter has MOVED INWARDS
(project (?name ?site)
(conditional
(sequence
(filter (> (strlen ?name) 8)
(bgp (triple ?shop rdf:type bs:Bookshop)
(triple ?shop rdfs:label ?name)))
(bgp (triple ?shop bs:locatedIn ?town))
(path ?town (path+ bs:within) bt:place-scotland))
(bgp (triple ?shop bs:website ?site))))
HOLOS -- the same pushdown, plus the join algorithms
Project(?name, ?site)
+- LeftJoin(HashBuildRightProbeLeft, keys = ?shop)
+- LeftJoin(HashBuildLeftProbeRight, keys = ?town)
| +- LeftJoin(HashBuildLeftProbeRight, keys = ?shop)
| | +- QuadPattern(?shop rdf:type bs:Bookshop)
| | +- Filter(STRLEN(?name) > 8)
| | +- QuadPattern(?shop rdfs:label ?name)
| +- QuadPattern(?shop bs:locatedIn ?town)
| +- Path(?town (bs:within)+ bt:place-scotland)
+- QuadPattern(?shop bs:website ?site)
Two independently written optimisers pushing the same
filter to the same place is a good sign the rewrite is
the right one.
Filter pushdown is the one to
recognise. In both plans the FILTER ends up directly on
the pattern that binds ?name, before the join and before the
path, so rows that can't survive are discarded as early as possible. If a
query is slow, compare the two forms: a filter that has not moved
usually can't, commonly because it mentions a variable the optimiser
can't prove is bound at that point.
13.0b
Reading any plan, and the debugging playbook
The operator names are shared across engines. So are the
failure modes.
What the operators mean
bgp / QuadPattern — a run of triple patterns. Ask how
many rows each matches alone (Q90).
join / sequence — two patterns sharing a variable. A
join with no key is the cross product of Q91.
leftjoin / conditional — OPTIONAL. Look
for a filter that ended up inside it (Q19).
filter — check how far down it was pushed. Further is
better.
path — rarely reordered, so put a selective pattern before
one.
group / extend — aggregation happens after the join,
so a multiplied join is already wrong by the time it runs (Q92).
slice — LIMIT, applied last, so it seldom saves
work unless the engine can push it.
THE RESULT IS EMPTY -- in payoff order
1 a datatype comparison Q07, Q62, Q60
gYear, date, wktLiteral -- go via STR()
2 a language tag Q95
"Cardiff" never equals "Cardiff"@en
3 a mistyped prefix or IRI Q96
http vs https, # vs /
4 a FILTER that escaped an OPTIONAL Q19
5 MINUS with no shared variable Q17
then bisect with Q93's EXISTS ladder.
FAR TOO MANY ROWS
1 count each pattern alone and MULTIPLY Q90, Q91
product matches your row count? nothing joined
2 look for a mistyped variable
3 or a legitimate one-to-many: two labels
per place doubles a count with nothing wrong
THE NUMBERS ARE WRONG, THE ROWS LOOK RIGHT
1 COUNT(DISTINCT ?x) != COUNT(?x)? something
multiplied before the aggregate Q92
2 SUM and AVG cannot be repaired with DISTINCT.
Split into sub-queries Q70
3 zero-count groups vanish entirely unless you
use OPTIONAL and COUNT(?x) Q25
IT IS SLOW
1 get the plan; check the filter was pushed down
2 run Q90 on your own predicates; does the join
order match the selectivity?
3 hunt for ?s ?p ?o, an unbound predicate, or a
path starting from an unbound variable
4 add a cheap selective pattern FIRST Q52
5 only then rewrite -- and measure, because the
fastest spelling depends on the store Q97
Q90
How selective is each pattern
Before optimising anything: how many rows does each pattern in my query match on its own?
# Q90 How selective is each pattern
# Before optimising anything: how many rows does each pattern in my query match on its own?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?pattern ?rows
WHERE {
{ { SELECT (COUNT(*) AS ?rows) WHERE { ?s bs:hasCafe true . } }
BIND( "?s bs:hasCafe true" AS ?pattern ) }
UNION
{ { SELECT (COUNT(*) AS ?rows) WHERE { ?s a bs:Bookshop . } }
BIND( "?s a bs:Bookshop" AS ?pattern ) }
UNION
{ { SELECT (COUNT(*) AS ?rows) WHERE { ?s bs:locatedIn ?o . } }
BIND( "?s bs:locatedIn ?o" AS ?pattern ) }
UNION
{ { SELECT (COUNT(*) AS ?rows) WHERE { ?s bs:stocks ?o . } }
BIND( "?s bs:stocks ?o" AS ?pattern ) }
UNION
{ { SELECT (COUNT(*) AS ?rows) WHERE { ?s rdfs:label ?o . } }
BIND( "?s rdfs:label ?o" AS ?pattern ) }
UNION
{ { SELECT (COUNT(*) AS ?rows) WHERE { ?s ?p ?o . } }
BIND( "?s ?p ?o (everything)" AS ?pattern ) }
}
ORDER BY ?rows
PREFIXbs:<https://example.org/bookshop-trail/schema#>PREFIXrdfs:<http://www.w3.org/2000/01/rdf-schema#>SELECT?pattern?rowsWHERE {
{ { SELECT (COUNT(*) AS?rows) WHERE { ?sbs:hasCafe true . } }
BIND( "?s bs:hasCafe true"AS?pattern ) }
UNION
{ { SELECT (COUNT(*) AS?rows) WHERE { ?sabs:Bookshop . } }
BIND( "?s a bs:Bookshop"AS?pattern ) }
UNION
{ { SELECT (COUNT(*) AS?rows) WHERE { ?sbs:locatedIn?o . } }
BIND( "?s bs:locatedIn ?o"AS?pattern ) }
UNION
{ { SELECT (COUNT(*) AS?rows) WHERE { ?sbs:stocks?o . } }
BIND( "?s bs:stocks ?o"AS?pattern ) }
UNION
{ { SELECT (COUNT(*) AS?rows) WHERE { ?srdfs:label?o . } }
BIND( "?s rdfs:label ?o"AS?pattern ) }
UNION
{ { SELECT (COUNT(*) AS?rows) WHERE { ?s?p?o . } }
BIND( "?s ?p ?o (everything)"AS?pattern ) }
}
ORDERBY?rows
How it works
An engine joins patterns in whatever order it thinks cheapest, and it decides using estimates. When it gets that wrong, the fix is usually to know the real numbers. Counting each pattern alone tells you which one is the filter and which one is the fan-out.
What to take away
Measure before optimising. An estimate that's wrong by a factor of a thousand is the usual cause of a slow query.
The most selective pattern should be evaluated first; the engine normally arranges that, and this is how you check.
`?s ?p ?o` matches the whole store. Never leave one in a query you care about the speed of.
count each pattern on its own, then read the spread:
+------------------------------------+--------+
| ?s bs:hasCafe true | 22 | <- selective
| ?s a bs:Bookshop | 33 |
| ?s bs:locatedIn ?o | 46 |
| ?s bs:stocks ?o | 95 |
| ?s rdfs:label ?o | 434 | <- fans out
| ?s ?p ?o | 4,698 |
+------------------------------------+--------+
A join costs roughly the product of what it joins, so the order
matters:
33 x 434 evaluated the wrong way round
33 -> 33 evaluated the selective pattern first
Most engines get this right unaided. When one does not, this
table tells you what to tell it -- and gives you the numbers to
argue with the plan in the README.
Editor6HOLOS6Fuseki6bookshop-trail-1.1.ttl
Q91
The cross product you did not mean to write
What happens when two patterns in the same group share no variable?
# Q91 The cross product you did not mean to write
# What happens when two patterns in the same group share no variable?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?joined ?crossed ?ratio
WHERE {
{
SELECT (COUNT(*) AS ?joined) WHERE {
?shop bs:locatedIn ?town .
?town rdfs:label ?name .
}
}
{
SELECT (COUNT(*) AS ?crossed) WHERE {
?shop bs:locatedIn ?town .
?other rdfs:label ?name .
}
}
BIND( ?crossed / ?joined AS ?ratio )
}
Two patterns in one group are joined. If they have no variable in common there's nothing to join on, so every row of one is paired with every row of the other. No error is raised. The result is simply enormous, and every aggregate over it's wrong.
What to take away
Patterns in the same group are joined on shared variables. With none shared, you get the Cartesian product and no warning.
The tell is that the row count equals the product of the pattern sizes measured separately. Count them with q90 and multiply.
The cause is nearly always a mistyped variable, which is why consistent naming pays for itself.
joined -- ?town is shared:
?shop bs:locatedIn ?town .
?town rdfs:label ?name .
^ ^
+-------+ the join
--> 63 rows
crossed -- one letter changed, and nothing is shared:
?shop bs:locatedIn ?town .
?other rdfs:label ?name .
-+---
+-- a different variable, so no join at all
--> 46 x 434 = 19,964 rows
+----------------------------------------------+
| the symptom: far too many rows, and every |
| COUNT and SUM inflated by the same factor |
| |
| the cause: almost always a typo in a |
| variable name |
+----------------------------------------------+
Spotting it: the crossed count is EXACTLY the product of the
two pattern sizes. Count each pattern alone, as in q90, and
multiply. If the answer matches, nothing joined.
Note that 19,964 / 63 is not a round number. The ratio is not
the tell -- the product is. The joined query returns 63 rather
than 46 because some towns carry two labels, so even the
correct query multiplies a little.
Editor1HOLOS1Fuseki1bookshop-trail-1.1.ttl
Q92
The join that inflates an aggregate
Why does counting shops per council area give the wrong number when events are in the same query?
# Q92 The join that inflates an aggregate
# Why does counting shops per council area give the wrong number when events are in the same query?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?councilName ?rows ?inflated ?correct ?events
WHERE {
?council a bs:CouncilArea ;
rdfs:label ?councilName .
{
SELECT ?council
(COUNT(*) AS ?rows)
(COUNT(?shop) AS ?inflated)
(COUNT(DISTINCT ?shop) AS ?correct)
(COUNT(DISTINCT ?event) AS ?events)
WHERE {
?shop a bs:Bookshop ; bs:locatedIn/bs:within+ ?council .
?council a bs:CouncilArea .
OPTIONAL { ?event bs:heldAt ?shop . }
}
GROUP BY ?council
}
FILTER( ?inflated != ?correct )
}
ORDER BY DESC(?inflated) ?councilName
Joining shops to events multiplies the shop rows: a shop with six events appears six times. COUNT(?shop) then counts appearances rather than shops. COUNT(DISTINCT ?shop) repairs the symptom; two sub-queries repair the cause.
What to take away
A join that multiplies rows corrupts every aggregate over them.
COUNT(DISTINCT ?x) survives it. SUM and AVG don't, and can't be made to.
If a query counts two unrelated things per group, it needs two aggregations -- not one join and some hope.
one pattern, two things being counted:
?shop bs:locatedIn/bs:within+ ?council .
OPTIONAL { ?event bs:heldAt ?shop }
North Yorkshire, 3 shops, 6 events:
Endpapers x 3 events -+
The Bookwyrm x 2 events -+- 6 rows, not 3
The Harbour Page x 1 -+
COUNT(?shop) = 6 wrong
COUNT(DISTINCT ?shop) = 3 right
COUNT(DISTINCT ?event) = 6 right
Greater London is starker still: 2 shops, 6 events, and
COUNT(?shop) reports 6 -- three times the true figure.
DISTINCT rescues COUNT. It does not rescue SUM or AVG:
summing floor area over those 6 rows counts Endpapers three
times, and no SUM(DISTINCT ...) means what you want, because
two shops may legitimately share a floor area.
The real fix is q70's: count each thing in its own sub-query,
and join the two summaries.
Editor15HOLOS15Fuseki15bookshop-trail-1.1.ttl
Q93
Why is my result empty
A query returns nothing. Which line is responsible?
# Q93 Why is my result empty
# A query returns nothing. Which line is responsible?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
SELECT ?step ?matches
WHERE {
{ BIND( "1 any bs:Bookshop at all" AS ?step )
BIND( EXISTS { ?s a bs:Bookshop . } AS ?matches ) }
UNION
{ BIND( "2 ...with a bs:founded" AS ?step )
BIND( EXISTS { ?s a bs:Bookshop ; bs:founded ?y . } AS ?matches ) }
UNION
{ BIND( "3 ...founded before 1970, cast directly" AS ?step )
BIND( EXISTS { ?s a bs:Bookshop ; bs:founded ?y .
FILTER( xsd:integer(?y) < 1970 ) } AS ?matches ) }
UNION
{ BIND( "4 ...founded before 1970, via STR()" AS ?step )
BIND( EXISTS { ?s a bs:Bookshop ; bs:founded ?y .
FILTER( xsd:integer(STR(?y)) < 1970 ) } AS ?matches ) }
}
ORDER BY ?step
Bisect it. Each branch tests a longer prefix of the query with EXISTS, so the answer flips from true to false at exactly the line that kills it. It beats commenting lines out by hand, and it runs in one go.
What to take away
Bisect with EXISTS rather than by commenting lines out. One run tells you where it breaks.
An empty result is far more often a datatype or language-tag mismatch than a genuine absence of data.
Nothing here errors. Silence is the normal failure mode in SPARQL, which is why a ladder like this is worth keeping to hand.
build the query up one line at a time, and ask EXISTS at each:
+-------------------------------------------+-------+
| 1 any bs:Bookshop at all | true |
| 2 ...with a bs:founded | true |
| 3 ...founded before 1970, cast directly | ??? | <- flips here
| 4 ...founded before 1970, via STR() | true |
+-------------------------------------------+-------+
Step 3 is the q07 trap: xsd:integer() applied straight to an
xsd:gYear. On Fuseki it is true; on the browser editor and on
HOLOS it is false, and no error is raised on any of them.
Run this on YOUR engine. Whichever line flips is the one to
rewrite -- and if none of them flips, the problem is in a part
of the query this ladder does not reach.
The empty-result checklist, in the order that pays off:
1 a datatype comparison (q07, q62, q60)
2 a language tag on a literal (q95)
3 a mistyped prefix or IRI (q96)
4 a FILTER that escaped an OPTIONAL (q19)
5 MINUS with no shared variable (q17)
Engines differ, and that's expected. step 3 is the point. Fuseki reports true, the browser editor and HOLOS report false, and none of them raises an error. That's what the query is for -- run it on the engine you actually use, and believe that column rather than this note.
Editor4HOLOS4Fuseki4bookshop-trail-1.1.ttl
Q94
Pin one case while you work on it
How do I run a complicated query against a single known resource?
# Q94 Pin one case while you work on it
# How do I run a complicated query against a single known resource?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?shopName ?townName ?countryName
WHERE {
# Delete this one line to run the query for real.
VALUES ?shop { bt:shop-cliff-road bt:shop-sea-margin }
?shop a bs:Bookshop ;
rdfs:label ?shopName ;
bs:locatedIn ?town .
?town rdfs:label ?townName .
?town bs:within+ ?country .
?country a bs:Country ;
rdfs:label ?countryName .
FILTER( LANG(?townName) = "en" && LANG(?countryName) = "en" )
}
ORDER BY ?shopName
PREFIXbt:<https://example.org/bookshop-trail/>PREFIXbs:<https://example.org/bookshop-trail/schema#>PREFIXrdfs:<http://www.w3.org/2000/01/rdf-schema#>SELECT?shopName?townName?countryNameWHERE {
# Delete this one line to run the query for real.VALUES?shop { bt:shop-cliff-roadbt:shop-sea-margin }
?shopabs:Bookshop ;
rdfs:label?shopName ;
bs:locatedIn?town .
?townrdfs:label?townName .
?townbs:within+ ?country .
?countryabs:Country ;
rdfs:label?countryName .
FILTER( LANG(?townName) = "en" && LANG(?countryName) = "en" )
}
ORDERBY?shopName
How it works
VALUES binds a variable to a fixed list before anything else runs, which turns a query over the whole dataset into a query over one row of it. Delete the line and the query is the real one again -- no other edits, so there's nothing to forget to undo.
What to take away
VALUES injects a fixed table of bindings, and is the cleanest way to pin a query to one case while you work on it.
It restricts before the join, so a clamped query stays fast however slow the unclamped one is.
One line in, one line out. Nothing else about the query changes, which is what makes it safe.
VALUES ?shop { bt:shop-cliff-road }
?shop a bs:Bookshop ; rdfs:label ?shopName ; bs:locatedIn ?town .
?town rdfs:label ?townName .
?town bs:within+ ?country .
?country a bs:Country ; rdfs:label ?countryName .
with the VALUES line 1 shop, easy to read
without it 33 shops, the real query
Add rows to widen the net without losing the focus:
VALUES ?shop { bt:shop-cliff-road
bt:shop-sea-margin
bt:shop-west-quay }
and several variables at once, when the interesting case is a
combination:
VALUES (?shop ?genre) {
(bt:shop-verso bt:genre-translated-fiction)
(bt:shop-quire bt:genre-graphic-novels)
}
Better than a FILTER for this: VALUES restricts before the join
rather than after it, so the debugging run is fast even when
the real query is not.
Editor2HOLOS2Fuseki2bookshop-trail-1.1.ttl
Q95
The language tag that stops a match
Why does comparing a label to a string find nothing?
# Q95 The language tag that stops a match
# Why does comparing a label to a string find nothing?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?comparison ?matches
WHERE {
{ { SELECT (COUNT(*) AS ?matches) WHERE {
?p a bs:Settlement ; rdfs:label ?label .
FILTER( ?label = "Cardiff" ) } }
BIND( "?label = Cardiff (untagged)" AS ?comparison ) }
UNION
{ { SELECT (COUNT(*) AS ?matches) WHERE {
?p a bs:Settlement ; rdfs:label ?label .
FILTER( ?label = "Cardiff"@en ) } }
BIND( "?label = Cardiff@en (tagged)" AS ?comparison ) }
UNION
{ { SELECT (COUNT(*) AS ?matches) WHERE {
?p a bs:Settlement ; rdfs:label ?label .
FILTER( STR(?label) = "Cardiff" ) } }
BIND( "STR(?label) = Cardiff (tag dropped)" AS ?comparison ) }
}
ORDER BY ?comparison
A literal with a language tag isn't equal to the same characters without one. They are different RDF terms. Nearly every label in this dataset is tagged, so the obvious comparison silently matches nothing at all.
What to take away
A language-tagged literal is a different term from the plain string, and equality between them is false.
STR() strips the tag and is the portable way to compare text.
langMatches(LANG(?l), "en") is the right test when you want a language rather than a value, and it handles subtags such as en-GB.
in the data: "Cardiff"@en tagged
in the query: "Cardiff" untagged
"Cardiff"@en = "Cardiff" --> false
three ways to write the comparison:
+--------------------------+------+------------------------+
| ?label = "Cardiff" | 0 | different terms |
| ?label = "Cardiff"@en | 1 | exact, but brittle |
| STR(?label) = "Cardiff" | 1 | drops the tag: robust |
+--------------------------+------+------------------------+
STR() is the general answer, and the same tool that fixes the
datatype traps in q07 and q62. It strips a literal to its
characters, whatever was attached.
Watch for this in ORDER BY too: sorting on a tagged literal is
not the same as sorting on its text, and engines differ on the
result (q25 found exactly that).
Editor3HOLOS3Fuseki3bookshop-trail-1.1.ttl
Q96
Find the mistyped IRI
A pattern matches nothing and the vocabulary looks right. How do I check?
# Q96 Find the mistyped IRI
# A pattern matches nothing and the vocabulary looks right. How do I check?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
SELECT ?namespace (COUNT(*) AS ?uses)
WHERE {
?s ?p ?o .
BIND( REPLACE(STR(?p), "[^#/]*$", "") AS ?namespace )
}
GROUP BY ?namespace
ORDER BY DESC(?uses)
Take a census of the predicate namespaces actually in use. A mistyped prefix produces an IRI in a namespace that appears nowhere else, or doesn't appear at all -- and either is obvious the moment the real namespaces are listed next to their counts.
What to take away
Listing the namespaces actually in use finds a mistyped prefix in one query.
A wrong IRI isn't an error in SPARQL. It's a pattern that matches nothing, which looks exactly like an empty dataset.
http versus https, and # versus /, are the two that catch everyone. Copy IRIs from the data; don't retype them.
strip each predicate back to its namespace and count:
REPLACE( STR(?p), "[^#/]*$", "" )
<...bookshop-trail/schema#founded> --> <...schema#>
+---------------------------------------------+-------+
| https://example.org/bookshop-trail/schema# | 2,248 |
| http://www.w3.org/1999/02/22-rdf-syntax-ns# | 1,088 |
| http://www.w3.org/2000/01/rdf-schema# | 620 |
| http://www.opengis.net/ont/geosparql# | 387 |
| http://www.w3.org/2004/02/skos/core# | 152 |
| http://www.w3.org/2003/01/geo/wgs84_pos# | 126 |
| http://purl.org/dc/terms/ | 74 |
| http://www.w3.org/2002/07/owl# | 3 |
+---------------------------------------------+-------+
Eight namespaces, and the counts are a sanity check in
themselves: owl# appears three times, because the vocabulary
declares exactly three OWL characteristics.
Now compare with what your query is asking for. The two
mistakes this catches:
schema: vs schema# one character, no match, no error
https: vs http: the same, and easier to miss
This dataset uses https://example.org/... and http://www.w3.org/...
-- both schemes, deliberately, because real data does.
The same census on rdf:type objects tells you the classes:
SELECT ?class (COUNT(*) AS ?n)
WHERE { ?s a ?class } GROUP BY ?class
Editor8HOLOS8Fuseki8bookshop-trail-1.1.ttl
Q97
Three spellings, one answer, three plans
Do these three ways of asking 'which shops are in Scotland' give the same result?
# Q97 Three spellings, one answer, three plans
# Do these three ways of asking 'which shops are in Scotland' give the same result?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
SELECT ?spelling ?shops
WHERE {
{ { SELECT (COUNT(DISTINCT ?shop) AS ?shops) WHERE {
?shop a bs:Bookshop ; bs:locatedIn ?town .
?town bs:within+ bt:place-scotland . } }
BIND( "A two patterns, path in the second" AS ?spelling ) }
UNION
{ { SELECT (COUNT(DISTINCT ?shop) AS ?shops) WHERE {
?shop a bs:Bookshop ; bs:locatedIn/bs:within+ bt:place-scotland . } }
BIND( "B one pattern, path chained on" AS ?spelling ) }
UNION
{ { SELECT (COUNT(DISTINCT ?shop) AS ?shops) WHERE {
bt:place-scotland ^bs:within+/^bs:locatedIn ?shop .
?shop a bs:Bookshop . } }
BIND( "C the same journey, backwards" AS ?spelling ) }
}
ORDER BY ?spelling
PREFIXbt:<https://example.org/bookshop-trail/>PREFIXbs:<https://example.org/bookshop-trail/schema#>SELECT?spelling?shopsWHERE {
{ { SELECT (COUNT(DISTINCT?shop) AS?shops) WHERE {
?shopabs:Bookshop ; bs:locatedIn?town .
?townbs:within+ bt:place-scotland . } }
BIND( "A two patterns, path in the second"AS?spelling ) }
UNION
{ { SELECT (COUNT(DISTINCT?shop) AS?shops) WHERE {
?shopabs:Bookshop ; bs:locatedIn/bs:within+ bt:place-scotland . } }
BIND( "B one pattern, path chained on"AS?spelling ) }
UNION
{ { SELECT (COUNT(DISTINCT?shop) AS?shops) WHERE {
bt:place-scotland ^bs:within+/^bs:locatedIn?shop .
?shopabs:Bookshop . } }
BIND( "C the same journey, backwards"AS?spelling ) }
}
ORDERBY?spelling
How it works
They do, and the engines build visibly different algebra for each. That's the point: a query is a description of what you want, not instructions for getting it, and the shape you write isn't necessarily the shape that runs. Run this, then read the plans in the module README.
What to take away
Several spellings of one question are common, and they are genuinely equivalent.
The engine rewrites what you wrote. Reading the plan is how you find out into what.
Write for the reader first. Rewrite for the optimiser only when you have measured a reason to.
A two patterns, path in the second
?shop bs:locatedIn ?town .
?town bs:within+ bt:place-scotland .
B one pattern, path chained on
?shop bs:locatedIn/bs:within+ bt:place-scotland .
C the same journey, walked backwards
bt:place-scotland ^bs:within+/^bs:locatedIn ?shop .
all three --> 9 shops
but the algebra differs. Jena, for A:
(sequence (bgp (triple ?shop bs:locatedIn ?town))
(path ?town (path+ bs:within) bt:place-scotland))
and for B it folds the join away entirely:
(path ?shop (seq bs:locatedIn (path+ bs:within)) bt:place-scotland)
Which is faster depends on the store, the data and the
direction the index favours. Measure; do not assume. What is
reliable is that all three mean the same thing -- so write the
one that reads best, and only reach for another when a
measurement tells you to.
Editor3HOLOS3Fuseki3bookshop-trail-1.1.ttl
Module 14
Putting it together
Questions with no single obvious shape, each needing two or three of the techniques above at once. Try them before reading the answer.
# Q69 Whose influence reaches furthest
# Which author has the largest number of literary descendants?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?authorName (COUNT(DISTINCT ?descendant) AS ?reach)
WHERE {
?author a bs:Author ; rdfs:label ?authorName .
OPTIONAL { ?author ^bs:influencedBy+ ?descendant . }
}
GROUP BY ?author ?authorName
ORDER BY DESC(?reach) ?authorName
LIMIT 15
Reachability plus aggregation. The inverse path ^bs:influencedBy+ runs down the influence graph from an author to everyone who inherits from them at any remove; counting the distinct endpoints ranks the authors. The two-cycle in the data means some authors appear among their own descendants, which is correct and worth noticing.
What to take away
Aggregating over a path result is how you measure a graph rather than just traverse it.
Reversing the path direction turns 'my ancestors' into 'my descendants' without touching the data.
COUNT(DISTINCT ...) matters here: several routes may reach the same descendant, and you want people, not paths.
influence points BACKWARDS -- from the later author to the earlier:
dilys-tremain --bs:influencedBy--> cerys-lloyd
so "who did X influence?" reverses it:
?ancestor ^bs:influencedBy+ ?descendant
---------+-------
one or more hops, downstream
+----------------+-------------+
| rhona-blackwood| many | the roots of the graph
| maud-ellery | many | reach almost everyone
| ... | |
| dilys-tremain | 0 | the leaves reach nobody
+----------------+-------------+
Watch for the mutual pair: tam-brodie and kirsty-lammond each
influenced the other, so each is among their own descendants.
COUNT(DISTINCT ?d) still terminates -- paths are cycle-safe.
Editor15HOLOS15Fuseki15bookshop-trail-1.1.ttl
Q70
A weekend in one county
For every council area, what's there to do: how many shops, how many events, and what do the shops specialise in?
# Q70 A weekend in one county
# For every council area, what's there to do: how many shops, how many events, and what do the shops specialise in?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
SELECT ?councilName ?shops ?events ?specialisms
WHERE {
{
SELECT ?council (COUNT(DISTINCT ?shop) AS ?shops)
(GROUP_CONCAT(DISTINCT ?genreName; SEPARATOR=", ") AS ?specialisms)
WHERE {
?shop a bs:Bookshop ;
bs:locatedIn/bs:within+ ?council ;
bs:specialises ?genre .
?council a bs:CouncilArea .
?genre skos:prefLabel ?genreName .
FILTER( LANG(?genreName) = "en" )
}
GROUP BY ?council
}
OPTIONAL {
SELECT ?council (COUNT(?e) AS ?events)
WHERE {
?e bs:heldAt ?s .
?s bs:locatedIn/bs:within+ ?council .
?council a bs:CouncilArea .
}
GROUP BY ?council
}
?council rdfs:label ?councilName .
}
ORDER BY DESC(?shops) ?councilName
Four techniques in one query. A property path climbs from shop to council area whatever the depth; a sub-query counts events without multiplying the shop count; GROUP_CONCAT lists the specialisms; and OPTIONAL keeps the council areas that have no shops at all.
What to take away
Counting two unrelated things per group needs two aggregations, not one join.
A join that multiplies rows corrupts every aggregate over it except COUNT(DISTINCT).
Build queries like this one piece at a time, checking the row count after each addition.
the trap this query avoids:
joining shops AND events in one pattern multiplies them --
a council with 3 shops and 8 events yields 24 rows, and
COUNT(DISTINCT ?shop) is then the only thing that still works
so events are counted in their own sub-query, per council:
+ outer ------------------------------------+
| ?shop bs:locatedIn/bs:within+ ?council |
| GROUP BY ?council |
| COUNT(DISTINCT ?shop) |
| GROUP_CONCAT(?specialism) |
| |
| + inner: events per council -----------+ |
| | ?e bs:heldAt ?s . | |
| | ?s bs:locatedIn/bs:within+ ?council | |
| | GROUP BY ?council | |
| +--------------------------------------+ |
+-------------------------------------------+
Counting two different things per group almost always means two
sub-queries. One join cannot serve both.
Engines differ, and that's expected. the order of the specialisms inside each GROUP_CONCAT is unspecified, and the three engines order them differently. The counts and the membership agree exactly. Same caveat as q24.
Editor21HOLOS21Fuseki21bookshop-trail-1.1.ttl
Q71
The gaps in the catalogue
Which genres does the trail specialise in but barely stock, and which does it stock without anyone specialising?
# Q71 The gaps in the catalogue
# Which genres does the trail specialise in but barely stock, and which does it stock without anyone specialising?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
SELECT ?genreName (COALESCE(?shops, 0) AS ?specialists)
(COALESCE(?works, 0) AS ?titles)
WHERE {
{ ?g ^bs:specialises ?someShop . }
UNION
{ ?someWork bs:genre ?g . }
?g skos:prefLabel ?genreName .
FILTER( LANG(?genreName) = "en" )
OPTIONAL {
SELECT ?g (COUNT(DISTINCT ?shop) AS ?shops)
WHERE { ?shop bs:specialises ?g . }
GROUP BY ?g
}
OPTIONAL {
SELECT ?g (COUNT(DISTINCT ?work) AS ?works)
WHERE { ?work bs:genre/skos:broader* ?g . }
GROUP BY ?g
}
}
ORDER BY ?specialists DESC(?titles) ?genreName
Two sets, compared. One sub-query counts the shops that name each genre as their specialism; another counts the works filed under it or anything narrower. A full outer comparison isn't available in SPARQL, so a UNION of the genres from both sides gives the key set, and OPTIONAL fills in whichever side is missing.
What to take away
SPARQL has no full outer join; UNION for the keys plus OPTIONAL for the values is how you build one.
COALESCE(?x, 0) converts an unbound value into something arithmetic and ORDER BY can use.
skos:broader* on the works side means a book filed under Tartan Noir counts towards Crime Fiction too.
SPARQL has no FULL OUTER JOIN. Build one:
step 1 -- every genre that appears on either side
{ ?g ^bs:specialises ?anyShop } shops' specialisms
UNION
{ ?w bs:genre/skos:broader* ?g } genres of works
step 2 -- OPTIONAL sub-query for each count
?g --+-- OPTIONAL { shops specialising } --> ?shops
+-- OPTIONAL { works in this genre } --> ?works
step 3 -- COALESCE turns "no match" into zero
COALESCE(?shops, 0)
+-------------------+-------+-------+
| genre | shops | works |
+-------------------+-------+-------+
| mountaineering | 1 | 2 | thin
| climate-fiction | 0 | 3 | stocked, nobody's speciality
| classics | 1 | 0 | claimed, nothing filed
+-------------------+-------+-------+
COALESCE is the tool for turning absence into a usable value.
Editor107HOLOS107Fuseki107bookshop-trail-1.1.ttl
Q72
Where should the next shop go
Rank the shopless towns by how underserved they are: population, and distance to the nearest existing shop.
Negation finds the towns without a shop; a sub-query finds each one's distance to the nearest shop on the National Grid; and a computed score combines the two. Nothing here's new -- it's q16, q54 and a BIND, assembled.
What to take away
Complex questions are assemblies of simple ones. Write and test the parts separately.
Scoring formulas belong in queries: they encode a policy, and policies change more often than facts.
State the units. A score mixing people and kilometres is meaningful only for ranking, and should never be reported as a quantity.
+ towns with no shop -------------+ q16
| FILTER NOT EXISTS { |
| ?s bs:locatedIn ?town } |
+------------+--------------------+
|
+ nearest shop, squared metres ---+ q54
| MIN( (dx)^2 + (dy)^2 ) |
+------------+--------------------+
|
+ combine ------------------------+
| score = population x km |
| ---+---- -+- |
| demand distance |
+---------------------------------+
A bigger town further from any shop scores higher. The formula
is a judgement, not a fact -- which is exactly why it belongs in
the query and not in the data.
Editor4HOLOS4Fuseki4bookshop-trail-full.ttl
Q73
A reading list from one shop
Starting at The Sea Margin, build a reading list: everything it stocks, plus everything by the authors who influenced those books' authors.
# Q73 A reading list from one shop
# Starting at The Sea Margin, build a reading list: everything it stocks, plus everything by the authors who influenced those books' authors.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT DISTINCT ?title ?authorName ?why
WHERE {
{
bt:shop-sea-margin bs:stocks ?work .
BIND( "in stock" AS ?why )
}
UNION
{
bt:shop-sea-margin bs:stocks ?stocked .
?stocked bs:author ?author .
?author bs:influencedBy+ ?ancestor .
?work bs:author ?ancestor .
BIND( "recommended: an influence on this shop's authors" AS ?why )
}
?work rdfs:label ?title .
?work bs:author ?writer .
?writer rdfs:label ?authorName .
}
ORDER BY ?why ?authorName ?title
PREFIXbt:<https://example.org/bookshop-trail/>PREFIXbs:<https://example.org/bookshop-trail/schema#>PREFIXrdfs:<http://www.w3.org/2000/01/rdf-schema#>SELECTDISTINCT?title?authorName?whyWHERE {
{
bt:shop-sea-marginbs:stocks?work .
BIND( "in stock"AS?why )
}
UNION
{
bt:shop-sea-marginbs:stocks?stocked .
?stockedbs:author?author .
?authorbs:influencedBy+ ?ancestor .
?workbs:author?ancestor .
BIND( "recommended: an influence on this shop's authors"AS?why )
}
?workrdfs:label?title .
?workbs:author?writer .
?writerrdfs:label?authorName .
}
ORDERBY?why?authorName?title
How it works
Two hops of a different kind, joined. From the shop to its stock is one link; from a book to its author's influences is a path; from those authors back to their works is an inverse. UNION keeps the shop's own stock and the wider recommendations in one list, labelled by where each came from.
What to take away
Chaining a forward link, a transitive path and an inverse link is the shape of most graph recommendations.
UNION with a labelling BIND keeps provenance in the result: the reader can see why each row is there.
This is what a graph database is for. The equivalent in SQL needs a recursive CTE and a good deal more typing.
branch 1 -- what the shop actually has
bt:shop-sea-margin --bs:stocks--> ?work
branch 2 -- what its authors were reading
bt:shop-sea-margin --bs:stocks--> ?stocked
| bs:author
v
?author
| bs:influencedBy+
v
?ancestor
| ^bs:author
v
?work
+--------------------+------------------+
| The Selkie Ledger | in stock |
| An Lochan | in stock |
| The Shieling | recommended |
| Cold Harbour | recommended |
+--------------------+------------------+
A recommendation engine in fourteen lines, and no machine
learning anywhere near it.
Editor23HOLOS23Fuseki23bookshop-trail-1.1.ttl
Q74
A trust report
Produce a report of every disputed fact in the dataset, the rival claims, and which source the evidence favours.
# Q74 A trust report
# Produce a report of every disputed fact in the dataset, the rival claims, and which source the evidence favours.
# Data: bookshop-trail-1.2.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?shopName
(GROUP_CONCAT(DISTINCT STR(?year); SEPARATOR=", ") AS ?claimedYears)
(SAMPLE(?accepted) AS ?accept)
(SAMPLE(?acceptedSource) AS ?onTheWordOf)
WHERE {
?shop bs:founded ?year {| bs:confidence ?c |} .
?shop rdfs:label ?shopName .
{
SELECT ?shop (MAX(?c2) AS ?bestConfidence)
WHERE { ?shop bs:founded ?y2 {| bs:confidence ?c2 |} }
GROUP BY ?shop
}
?shop bs:founded ?accepted
{| bs:confidence ?bestConfidence ; bs:claimedBy ?src |} .
?src rdfs:label ?acceptedSource .
}
GROUP BY ?shop ?shopName
HAVING( COUNT(DISTINCT ?year) > 1 )
ORDER BY ?shopName
The capstone. Annotations supply the claims, a sub-query finds the best-supported one, GROUP_CONCAT lists the rivals, and the result is a single table a human could act on. It uses most of the course at once, which is the point.
What to take away
RDF 1.2 annotations plus ordinary SPARQL aggregation produce real provenance reporting, with no special machinery.
HAVING on a count is how you keep only the interesting groups.
The dataset stores what was claimed; the query decides what to believe. Keeping those separate is the whole argument for statement-level annotation.
for every shop with more than one founding claim:
+ all claims ----------------------------+
| ?shop bs:founded ?y {| claimedBy ?s ; |
| confidence ?c |}|
+------------+---------------------------+
|
+ best per shop ---------+ + list them all ----------+
| MAX(?c) -> ?best | | GROUP_CONCAT(?y) |
+------------+-----------+ +------------+------------+
+-----------+---------------+
v
+--------------+----------+----------+-------------------+
| shop | claims | accepted | on the word of |
+--------------+----------+----------+-------------------+
| Ex Libris |1919,1921 | 1919 | national register |
| Endpapers |1946,1949 | 1949 | national register |
| Candlemas |1928,1931 | 1931 | national register |
| Castle Steps |1962,1965 | 1962 | trail guide 2024 |
+--------------+----------+----------+-------------------+
HAVING(COUNT(DISTINCT ?y) > 1) keeps only the genuine
disagreements: shops whose sources agree are not news.
Engines differ, and that's expected. the years inside the GROUP_CONCAT come out in different orders on the three engines, and SAMPLE is explicitly allowed to pick any value from its group. The shops listed and the year accepted for each are identical everywhere.
Editor4HOLOS4Fuseki4bookshop-trail-1.2.ttl
Module 15
Beyond the standard
Reference rather than lesson. Every engine adds functions the specification doesn't define -- ARQ's afn:, SPIN's spif:, the XPath fn: library, GeoSPARQL's geof: -- and they're genuinely useful right up until you move the query. This module measures which of them your three engines actually have, shows what each does when a function is missing, and ends with the portable rewrite.
# Q100 Readable names without a label
# Show each shop's IRI as a short name, without joining to rdfs:label.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX afn: <http://jena.apache.org/ARQ/function#>
SELECT ?localName ?namespace ?name
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name .
BIND( afn:localname(?shop) AS ?localName )
BIND( afn:namespace(?shop) AS ?namespace )
}
ORDER BY ?localName
LIMIT 8
afn:localname splits an IRI at its last slash or hash and returns the tail; afn:namespace returns the rest. Both come from ARQ, Jena's query engine, and neither is in the SPARQL specification. They are the most useful of the extensions and the easiest to become dependent on.
What to take away
afn: is ARQ's function library. Jena has it, HOLOS implements it too, and it is not part of SPARQL.
afn:localname and afn:namespace take an IRI apart, which is otherwise a REPLACE with a regular expression -- see q104.
Reach for an extension when it makes a query clearer, but know you have done it. Q104 is the way back out.
<https://example.org/bookshop-trail/shop-inkwell>
-------------------+--------------- ------+-----
| |
afn:namespace ---' '--- afn:localname
"https://example.org/bookshop-trail/" "shop-inkwell"
what it's for: a diagnostic listing where the IRIs matter and
the labels would just get in the way -- q96's namespace census
is the same idea done with REPLACE.
where it works, measured:
afn:localname editor error holos yes fuseki yes
afn:namespace editor error holos yes fuseki yes
Comunica raises rather than returning nothing, which is the
better of the two failure modes: you find out.
Note. Comunica has no afn: functions and raises an error rather than returning unbound, so the editor is not claimed here.
EditorHOLOS8Fuseki8bookshop-trail-1.1.ttl
Q101
The square root module 09 could not have
How far is each shop from York, in actual kilometres?
Module 09 ranked shops by squared distance because SPARQL 1.1 has no square root. afn:sqrt supplies one. The arithmetic is the same as q56's; the only difference is that the answer is now a distance rather than an ordering.
What to take away
afn:sqrt, afn:pi, afn:e, afn:min and afn:max fill the gaps in SPARQL's arithmetic.
Fuseki also has the XPath math: library -- math:sqrt, math:pow, math:log -- which HOLOS does not. Two engines, two different sets of extras.
A ranking rarely needs the root. Take it only when you are going to show the number to somebody.
module 09, portable: here, with an extension:
d2 = dx*dx + dy*dy d = afn:sqrt(dx*dx + dy*dy)
ORDER BY ?d2 ORDER BY ?d
ranks correctly ranks correctly AND
but the number is gives the distance
metres squared
+------------------+---------+
| Endpapers | 0.5 km|
| The Bookwyrm | 0.6 km|
| The Harbour Page | 66.1 km|
| Cotton Quarto | 93.7 km|
+------------------+---------+
measured: afn:sqrt editor error holos yes fuseki yes
Note what module 09 bought by not using this: the same query,
without the square root, runs in the browser too. That is the
trade in one line.
EditorHOLOS8Fuseki8bookshop-trail-full.ttl
Q135
The XPath maths library
Compute a distance with math:sqrt and a growth figure with math:pow.
math: is the XPath 3.1 maths library, and Jena registers it. It covers what afn: covers and rather more -- pow, log, exp, the trigonometric functions -- and unlike afn: it is at least a published specification, even if not one SPARQL requires.
What to take away
math: is the XPath maths library: sqrt, pow, log, exp and the trigonometric functions. Jena has it; the other two do not.
It overlaps afn: without replacing it, and afn: is available on more of these engines.
Four libraries, three engines, no agreement. That is the state of extension functions, and it is why the portable rewrite matters.
math:sqrt(x) math:pow(x, y) math:exp(x)
math:log(x) math:log10(x) math:sin cos tan
math:atan2(y, x) math:pi()
measured on the three engines:
math:sqrt editor error holos error fuseki yes
math:pow editor error holos error fuseki yes
So this is the narrowest of the extension libraries in this
course: one engine of the three. afn:sqrt (q101) covers the
same ground on two of them, which makes afn: the better bet
if you are going to depend on an extension at all.
+----------------------------------------------------------+
| What to take from module 15 as a whole: there are four |
| function libraries here and no two engines agree on |
| which they have. Write the standard version (q104), |
| measure whether it is fast enough, and only then reach |
| for a library -- with a comment saying which engine you |
| have just tied yourself to. |
+----------------------------------------------------------+
Note. Fuseki only. Comunica and HOLOS both raise on math:, so the portable version of this is q104 and the two-engine version is q101.
EditorHOLOSFuseki8bookshop-trail-full.ttl
Q102
SPIN's string functions, and a silent failure
Tidy up some strings with spif:, and find out what happens where spif: is not implemented.
# Q102 SPIN's string functions, and a silent failure
# Tidy up some strings with spif:, and find out what happens where spif: is not implemented.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX spif: <http://spinrdf.org/spif#>
SELECT ?name ?titled ?trimmed ?shopAt
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name .
BIND( spif:titleCase( LCASE(STR(?name)) ) AS ?titled )
BIND( spif:trim( CONCAT(" ", STR(?name), " ") ) AS ?trimmed )
BIND( spif:indexOf( STR(?name), "o" ) AS ?shopAt )
}
ORDER BY ?name
LIMIT 8
SPIN's spif: library has the string helpers SPARQL never grew: trim, titleCase, indexOf, buildString. HOLOS implements them. Jena does not -- and rather than complaining, it returns the row with the variable unbound, which is the failure mode this course keeps warning about.
What to take away
spif: is SPIN's function library. HOLOS has it; Jena and Comunica do not.
An unimplemented function is not guaranteed to be an error. Jena leaves the variable unbound and returns the row, which is indistinguishable from the data simply not being there.
If a column comes back empty, suspect the function before you suspect the data.
spif:titleCase("the inkwell") -> "The Inkwell"
spif:trim(" x ") -> "x"
spif:indexOf("bookshop","shop") -> 4
spif:buildString("{?1}-{?2}", "a", "b") -> "a-b"
measured, and this is the point of the query:
spif:trim editor ERROR holos yes fuseki UNBOUND
spif:titleCase editor ERROR holos yes fuseki UNBOUND
spif:indexOf editor ERROR holos yes fuseki UNBOUND
+----------+-------------------------------------------+
| Comunica | raises. You find out immediately. |
| Jena | returns the row, variable unbound, HTTP |
| | 200. Looks exactly like missing data. |
| HOLOS | answers. |
+----------+-------------------------------------------+
Same shape as geof:distance in metres on Jena (q57), and the
reason the checking harness compares values rather than counting
rows: a column of blanks and a column of answers both have the
same number of rows.
Note. HOLOS only. Jena parses this and returns eight rows with three empty columns; Comunica raises. Run it on Fuseki yourself -- seeing the blanks is worth more than reading about them.
EditorHOLOS8Fusekibookshop-trail-1.1.ttl
Q103
The XPath library, under different names
Do the same string work with fn: instead of the SPARQL built-ins.
# Q103 The XPath library, under different names
# Do the same string work with fn: instead of the SPARQL built-ins.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX fn: <http://www.w3.org/2005/xpath-functions#>
SELECT ?name ?upper ?length ?firstFour
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name .
BIND( fn:upper-case( STR(?name) ) AS ?upper )
BIND( fn:string-length( STR(?name) ) AS ?length )
BIND( fn:substring( STR(?name), 1, 4 ) AS ?firstFour )
}
ORDER BY ?name
LIMIT 8
The fn: library is XPath and XQuery Functions and Operators, which SPARQL borrowed from without adopting wholesale. Several fn: functions duplicate a SPARQL built-in exactly, and the built-in is the one to use -- but you will meet fn: in other people's queries, and the indexing differs in a way that bites.
What to take away
fn: is the XPath function library. Where a SPARQL built-in does the same job, use the built-in.
Two functions that look equivalent may index differently: fn:substring counts from 1 and takes a length, afn:substr counts from 0 and takes an end position.
Knowing fn: is for reading other people's queries. Writing it into your own only costs you portability.
the same operation, two spellings:
UCASE(?s) fn:upper-case(?s)
STRLEN(?s) fn:string-length(?s)
SUBSTR(?s, 1, 4) fn:substring(?s, 1, 4)
CONTAINS(?s, "x") fn:contains(?s, "x")
YEAR(?d) fn:year-from-dateTime(?d)
and one that is NOT the same:
fn:substring is 1-based, like SUBSTR
afn:substr is 0-based, with an END index, not a length
SUBSTR("bookshop", 1, 4) -> "book"
afn:substr("bookshop", 0, 4) -> "book"
^ ^
| '-- end, not length
'----- counts from zero
measured: fn: editor error holos yes fuseki yes
Prefer the built-in every time. It is shorter, it is in the
specification, and it runs in the browser.
EditorHOLOS8Fuseki8bookshop-trail-1.1.ttl
Q104
The same query, with nothing but the standard
Get the local name, the namespace and a real distance using only SPARQL 1.1.
# Q104 The same query, with nothing but the standard
# Get the local name, the namespace and a real distance using only SPARQL 1.1.
# Data: bookshop-trail-full.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
SELECT ?localName ?namespace ?squaredMetres
WHERE {
bt:place-york bs:easting ?ye ; bs:northing ?yn .
?shop a bs:Bookshop ;
bs:easting ?se ;
bs:northing ?sn .
BIND( REPLACE(STR(?shop), "^.*[/#]", "") AS ?localName )
BIND( REPLACE(STR(?shop), "[^/#]*$", "") AS ?namespace )
BIND( ROUND( (?se - ?ye) * (?se - ?ye)
+ (?sn - ?yn) * (?sn - ?yn) ) AS ?squaredMetres )
}
ORDER BY ?squaredMetres
LIMIT 8
Everything the previous four queries needed an extension for, done with built-ins. REPLACE takes an IRI apart. A square root is unavailable, so the distance stays squared -- and the row is ordered correctly regardless, because that is all a ranking needs.
What to take away
REPLACE with a small regular expression replaces afn:localname and afn:namespace, and runs everywhere.
Some extensions have no standard equivalent at all -- a square root is the clearest -- and then the question is whether you need the value or only the order.
Write the standard version first. Reach for an extension when the standard one is genuinely worse, and leave a comment saying which engine you have just tied the query to.
afn:localname(?iri)
-> REPLACE(STR(?iri), "^.*[/#]", "")
afn:namespace(?iri)
-> REPLACE(STR(?iri), "[^/#]*$", "")
spif:titleCase(?s)
-> no built-in. CONCAT(UCASE(SUBSTR(?s,1,1)), SUBSTR(?s,2))
does the first word, which is usually what was meant.
afn:sqrt(?d2)
-> nothing. Rank on ?d2 instead, or compare against a
squared threshold (q53, q56).
+----------------------------------------------------------+
| runs on the editor, HOLOS and Fuseki |
| no library, no engine lock-in, no silent unbound column |
+----------------------------------------------------------+
That is the trade this module exists to show. The extensions
are real and they are useful; the standard is the thing that
travels.
Editor8HOLOS8Fuseki8bookshop-trail-full.ttl
Module 16
Blank nodes
The nodes with no name. They are how RDF writes lists, restrictions and anything else that is structure rather than a thing, and they behave differently from everything else in the language: they have identity inside a query and none outside it. This module is late in the sequence because it needs property paths and sub-queries, but the hazards in it turn up from module 01 onwards.
# Q109 Where the blank nodes are
# This dataset has blank nodes in it. Which predicates do they carry?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
SELECT ?p (COUNT(*) AS ?triples)
WHERE {
?b ?p ?o .
FILTER( isBLANK(?b) )
}
GROUP BY ?p
ORDER BY DESC(?triples) ?p
isBLANK is the test. Grouping the triples whose subject is blank by predicate gives a census: it says what the blank nodes are being used for without needing to name any of them.
What to take away
isBLANK(?x) is true when a term is a blank node -- a node with no IRI, which exists only inside the graph that contains it.
A census by predicate is the quickest way to find out what the blank nodes in an unfamiliar dataset are doing.
Blank nodes cluster in structure: lists, restrictions, shapes. Data about real things usually has IRIs, and should.
?b ?p ?o . FILTER( isBLANK(?b) )
-- ---------
| keeps the rows whose subject has no IRI
'-- binds to a blank node, but you can never write its name
the census, all four thousand-odd triples in:
rdf:first 19 the list cells that hold a member
rdf:rest 19 the cells that point to the next one
rdf:type 4 the two disjointness axioms
plus two anonymous owl:Class nodes
owl:members 2
owl:unionOf 2
----
46 triples, on 23 blank nodes
All of them are in 01-vocabulary.ttl. That is typical: blank
nodes cluster in schema and in structures -- lists, restrictions,
addresses, SHACL shapes -- and are rare in plain instance data,
where things deserve names.
isIRI, isLITERAL and isNUMERIC are the other three tests, and
they partition every RDF term between them.
Editor5HOLOS5Fuseki5bookshop-trail-1.1.ttl
Q110
The label in your results is not a name
A previous query returned _:b0. What happens if you put _:b0 back into a query?
# Q110 The label in your results is not a name
# A previous query returned _:b0. What happens if you put _:b0 back into a query?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
SELECT (COUNT(*) AS ?triples)
WHERE {
_:b0 ?p ?o .
}
SELECT (COUNT(*) AS?triples)
WHERE {
_:b0 ?p?o .
}
How it works
Nothing you would want. A blank node label written in a query pattern is not a reference to anything -- it is a variable that you are not allowed to project. It matches every term in the graph, IRIs and literals included, so this query counts the whole dataset.
What to take away
_:label in a query pattern is a variable, not a reference. It matches anything and cannot be projected.
Blank node labels in results are made up by the engine per query. They are not identifiers and will not survive a second query.
The way back to a blank node is a path from a named node, or a skolem IRI you mint yourself.
SELECT (COUNT(*) AS ?triples) WHERE { _:b0 ?p ?o }
4826
----
every triple in the file
What you probably meant: What SPARQL heard:
"the node called _:b0" "some subject, I don't care
which, and don't ask me to
show it to you"
_:b0 there behaves exactly like ?anything, minus the ability
to SELECT it. The label is scoped to the query, and it has no
connection to the _:b0 an earlier query printed -- or to the
_:b0 in the Turtle file, come to that.
So the rule is:
+--------------------------------------------------------+
| A blank node label in query results is a local nickname |
| the engine made up while answering. It is not stable |
| between queries, between engines, or between runs. |
| You cannot look one up. Do not store one. |
+--------------------------------------------------------+
Which leaves a real problem: how do you get back to a blank
node you have found? Two answers -- reach it by a path from
something that does have an IRI (q111), or give it a name of
your own (q114).
Note. 4826 is every triple in bookshop-trail-1.1.ttl, which is the point: _:b0 constrained nothing at all.
Editor1HOLOS1Fuseki1bookshop-trail-1.1.ttl
Q111
Walking an RDF collection
Which classes are declared disjoint from one another?
# Q111 Walking an RDF collection
# Which classes are declared disjoint from one another?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
SELECT ?member
WHERE {
?axiom a owl:AllDisjointClasses ;
owl:members/rdf:rest*/rdf:first ?member .
}
ORDER BY ?member
An RDF collection -- the ( a b c ) in Turtle -- is not a list structure the query language knows about. It is a chain of blank nodes, each carrying rdf:first for its item and rdf:rest for the rest of the chain. rdf:rest*/rdf:first is the path that walks it, and it is the reason property paths and blank nodes belong in the same lesson.
What to take away
An RDF collection is a chain of blank nodes with rdf:first and rdf:rest. Nothing in SPARQL treats it as a list.
rdf:rest*/rdf:first is the idiom for reading one out. The star, not the plus -- the plus drops the first member.
This is also why you can reach a blank node without naming it: start from something with an IRI and walk.
what the file says:
[] a owl:AllDisjointClasses ;
owl:members ( bs:Settlement bs:CouncilArea bs:Region bs:Country ) .
what is actually stored:
_:axiom --owl:members--> _:c1 --rdf:first--> bs:Settlement
|
rdf:rest
v
_:c2 --rdf:first--> bs:CouncilArea
|
rdf:rest
v
_:c3 --rdf:first--> bs:Region
|
rdf:rest
v
_:c4 --rdf:first--> bs:Country
|
rdf:rest
v
rdf:nil <- the end of the list
the path that gets the members out:
owl:members / rdf:rest* / rdf:first
----------- --------- ---------
one hop to zero or the item
the first more hops in this
cell along the cell
chain
Note rdf:rest* and not rdf:rest+ : the zero-length case is what
lets the first member out. With + you would silently lose it.
Fifteen classes come back, from the two axioms in the file.
Editor15HOLOS15Fuseki15bookshop-trail-1.1.ttl
Q112
Where in the list?
An RDF collection is ordered. Which position does each member hold?
# Q112 Where in the list?
# An RDF collection is ordered. Which position does each member hold?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
SELECT ?member (COUNT(?between) AS ?position)
WHERE {
?axiom a owl:AllDisjointClasses ;
owl:members ?head .
?head rdf:rest*/rdf:first bs:Country .
?head rdf:rest* ?between .
?between rdf:rest* ?cell .
?cell rdf:first ?member .
}
GROUP BY ?cell ?member
ORDER BY ?position
The order is in the chain, not in any property, so it has to be counted. Every cell from the head to this one is one hop of rdf:rest*, so counting those cells gives the position. It is an aggregate over a path, which is a shape worth having in your hands.
What to take away
RDF collections are ordered, and the order lives in the chain. Counting rdf:rest* hops is how you recover it.
GROUP BY on a blank node works and is deterministic within a single query. Sorting or paging on one is not -- see the note on q64.
Counting along a path is a general trick: depth in a hierarchy is the same query with a different predicate.
head --> c1 --> c2 --> c3 --> c4 --> nil
for c3, how many cells lie between the head and it, inclusive?
head rdf:rest* ?between . <- c1, c2, c3, c4 (and head=c1)
?between rdf:rest* c3 . <- keeps c1, c2, c3
-------------
COUNT = 3
so ?position falls out of a GROUP BY on the cell:
+--------------------+----------+
| bs:Settlement | 1 |
| bs:CouncilArea | 2 |
| bs:Region | 3 |
| bs:Country | 4 |
+--------------------+----------+
GROUP BY ?cell groups on a blank node, which is fine: within
one query the engine knows perfectly well which node is which.
It is only outside the query that the identity evaporates.
The FILTER on bs:Country picks the four-member axiom out of the
two in the file, using a member as the handle -- again, reaching
a blank node through something named.
Editor4HOLOS4Fuseki4bookshop-trail-1.1.ttl
Q113
Telling two blank nodes apart
Are the two disjointness axioms really two nodes, or one node found twice?
# Q113 Telling two blank nodes apart
# Are the two disjointness axioms really two nodes, or one node found twice?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX owl: <http://www.w3.org/2002/07/owl#>
SELECT (COUNT(*) AS ?orderedPairs)
WHERE {
?a a owl:AllDisjointClasses .
?b a owl:AllDisjointClasses .
FILTER( !sameTerm(?a, ?b) )
}
Blank nodes have identity inside a query even though they have no name outside it. sameTerm compares two bindings and answers truthfully, so a self-join with a not-same filter counts the distinct pairs.
What to take away
sameTerm compares RDF terms by identity, and it is the right test for blank nodes.
Identity is real within a query and gone outside it. That is the whole of what makes blank nodes awkward.
[] is an unnamed variable used once. Two [] in one query are two different variables, not the same node.
?a a owl:AllDisjointClasses .
?b a owl:AllDisjointClasses .
FILTER( !sameTerm(?a, ?b) )
with two axioms in the file:
(a1,a1) (a1,a2) the filter removes the diagonal
(a2,a1) (a2,a2) and 2 ordered pairs survive
+--------------------------------------------------------+
| inside one query blank nodes have identity. They |
| join, compare and group correctly. |
| |
| outside the query they have none. The label you saw |
| means nothing to the next query. |
+--------------------------------------------------------+
A note on the anonymous form: writing
[] a owl:AllDisjointClasses ; owl:members ?head .
means exactly what ?axiom meant in q111 -- one node, matched
once, just not projectable. Two separate [] in the same query
are two separate variables, and nothing stops them matching the
same node.
Use != rather than !sameTerm and most engines will still do the
right thing on blank nodes, but sameTerm is the operator that is
actually defined for them, and it does not raise on terms that
cannot be compared by value.
Editor1HOLOS1Fuseki1bookshop-trail-1.1.ttl
Q114
Giving a blank node a name
Mint a stable IRI for each disjointness axiom, so it can be quoted in a bug report.
# Q114 Giving a blank node a name
# Mint a stable IRI for each disjointness axiom, so it can be quoted in a bug report.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
SELECT ?id ?size ?firstMember
WHERE {
{
SELECT ?axiom (COUNT(?m) AS ?size) (MIN(?local) AS ?firstMember)
WHERE {
?axiom a owl:AllDisjointClasses ;
owl:members/rdf:rest*/rdf:first ?m .
BIND( REPLACE(STR(?m), "^.*[/#]", "") AS ?local )
}
GROUP BY ?axiom
}
BIND( IRI(CONCAT("https://example.org/bookshop-trail/axiom-",
LCASE(?firstMember), "-", STR(?size))) AS ?id )
}
ORDER BY ?id
Skolemising means replacing a blank node with an IRI. The IRI has to come from the node's content, never from its label, because the label changes between runs. Here the content is the size of the axiom and its alphabetically first member, computed in a sub-query and turned into an IRI with CONCAT and IRI().
What to take away
Skolemising replaces a blank node with an IRI so it can be referred to from outside. Derive the IRI from content, never from the label.
IRI(CONCAT(...)) builds an IRI at query time; the .well-known/genid/ prefix is the published convention for skolem IRIs.
BNODE() is the opposite operation, and it mints a fresh node per solution -- useful for structure, useless for identity.
a blank node an IRI you can send to somebody
------------ ------------------------------
_:b0 --skolemise--> bt:axiom-bookshop-11
_:b1 bt:axiom-councilarea-4
the key must come from the CONTENT:
MIN(?local) -> "Bookshop" stable across runs
COUNT(?m) -> 11 stable across engines
STR(?axiom) -> "b0" CHANGES. Never use it.
+--------------------------------------------------------+
| RDF has a convention for this: IRIs under |
| .../.well-known/genid/ are understood to be |
| skolem IRIs -- IRIs standing in for blank nodes. |
| A readable name works just as well when the graph is |
| yours; the convention matters when you publish. |
+--------------------------------------------------------+
Going the other way, BNODE() mints a fresh blank node -- one
per solution, so it is the tool for building structure in a
CONSTRUCT template rather than for identifying anything.
Skolemising is not free. You have asserted that these two
axioms are distinct, findable things, and if the file changes
so that a third axiom also starts with Bookshop and has eleven
members, two of them collide. Pick a key that cannot.
Editor2HOLOS2Fuseki2bookshop-trail-1.1.ttl
Q136
Names you can compute
Mint identifiers three ways -- a hash, a URI-safe string, and a fresh blank node -- and see which are stable.
# Q136 Names you can compute
# Mint identifiers three ways -- a hash, a URI-safe string, and a fresh blank node -- and see which are stable.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?name ?digest ?safe ?uuidLength ?struuidLength
WHERE {
?shop a bs:Bookshop ;
rdfs:label ?name .
BIND( SUBSTR( SHA256( STR(?name) ), 1, 12 ) AS ?digest )
BIND( ENCODE_FOR_URI( STR(?name) ) AS ?safe )
BIND( STRLEN( STR( UUID() ) ) AS ?uuidLength )
BIND( STRLEN( STRUUID() ) AS ?struuidLength )
BIND( BNODE() AS ?fresh )
FILTER( isBLANK(?fresh) )
}
ORDER BY ?name
LIMIT 6
q114 skolemised by hand. These are the functions for doing it properly: MD5 and the SHA family hash a string to a fixed-length digest, ENCODE_FOR_URI makes an arbitrary string safe to put in an IRI, and BNODE() mints a fresh blank node. Only the first two are repeatable, and that is the entire lesson.
What to take away
MD5, SHA1, SHA256, SHA384 and SHA512 hash a string. The digest is stable, so an IRI built from one is repeatable.
ENCODE_FOR_URI makes a string safe for an IRI path. It is the readable alternative to a hash.
BNODE(), UUID() and STRUUID() produce something new every time. Never use them where a re-run must produce the same graph.
"The Harbour Page"
MD5(?s) d41d8cd9... 32 hex characters
SHA256(?s) e3b0c442... 64
ENCODE_FOR_URI(?s) The%20Harbour%20Page
BNODE() _:b0, _:b1, ... a new one each solution
STRUUID() 36 characters, random
UUID() a urn:uuid: IRI, 45 characters
+---------------------+------------+---------------------------+
| | repeatable | good for |
+---------------------+------------+---------------------------+
| MD5 / SHA256 | yes | an id derived from content|
| ENCODE_FOR_URI | yes | a readable id from a label|
| BNODE() | NO | structure in a template |
| UUID / STRUUID | NO | a genuinely new thing |
+---------------------+------------+---------------------------+
Repeatable is what makes a migration re-runnable (q121). Run
the same INSERT twice with a hashed IRI and the second run
changes nothing; run it with UUID() and you get a second copy
of everything.
This query asks for the LENGTH of the random values rather
than the values, because a random value cannot be compared
across three engines -- which is itself the point being made.
MD5 is fine here and is not fine for anything security-shaped.
SHA256 costs nothing extra; prefer it by habit.
Editor6HOLOS6Fuseki6bookshop-trail-1.1.ttl
Q115
A copy with no blank nodes left in it
Build a graph that says the same thing about disjointness, with every blank node replaced by something nameable.
# Q115 A copy with no blank nodes left in it
# Build a graph that says the same thing about disjointness, with every blank node replaced by something nameable.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
CONSTRUCT {
?id bs:disjointMember ?member .
}
WHERE {
{
SELECT ?axiom (COUNT(?m) AS ?size) (MIN(?local) AS ?firstMember)
WHERE {
?axiom a owl:AllDisjointClasses ;
owl:members/rdf:rest*/rdf:first ?m .
BIND( REPLACE(STR(?m), "^.*[/#]", "") AS ?local )
}
GROUP BY ?axiom
}
?axiom owl:members/rdf:rest*/rdf:first ?member .
BIND( IRI(CONCAT("https://example.org/bookshop-trail/axiom-",
LCASE(?firstMember), "-", STR(?size))) AS ?id )
}
CONSTRUCT with a skolem IRI in the subject position. The list structure disappears: instead of a chain of cells, each axiom gets a flat set of bs:disjointMember triples. The result loads into any engine, survives a round trip through a file, and can be diffed.
What to take away
CONSTRUCT plus a skolem IRI is how you hand a blank-node structure to something that cannot cope with blank nodes.
Flattening a collection loses its order and its meaning as a single axiom. Know what you are giving up.
A graph with no blank nodes can be diffed, merged and quoted. That is worth a lot when the data is under review.
before -- 23 blank nodes, 46 triples of plumbing:
_:a --owl:members--> _:c1 --rdf:rest--> _:c2 --rdf:rest--> ...
| |
rdf:first rdf:first
v v
bs:Settlement bs:CouncilArea
after -- no blank nodes, no plumbing:
bt:axiom-bookshop-11 bs:disjointMember bs:Bookshop ;
bs:disjointMember bs:Person ;
...
bt:axiom-councilarea-4 bs:disjointMember bs:Settlement ;
...
15 triples out, one per member, and every subject has a name.
What was lost: the order of the list, and the fact that OWL
reads it as one axiom rather than a bag of memberships. That is
the trade -- flattening is easier to query and no longer means
quite the same thing. Do it for reporting and for diffing, not
as a replacement for the original.
Sending the result somewhere: press Get All in the editor after
running a CONSTRUCT and the constructed triples come back as
Turtle you can save.
Editor15HOLOS15Fuseki15bookshop-trail-1.1.ttl
Module 17
Updating the data
Everything before this reads. SPARQL Update writes: INSERT, DELETE, the two together, whole-graph operations, and a migration applied in place rather than handed back. An update returns nothing, so every query here comes with a second one that shows what it did. The editor's SPARQL panel cannot run these -- use Fuseki or HOLOS.
# Q116 Adding facts you already know
# Add a new bookshop to the trail.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
INSERT DATA {
bt:shop-foxed-page
a bs:Bookshop ;
rdfs:label "The Foxed Page"@en ;
bs:locatedIn bt:place-hay-on-wye ;
bs:founded "2019"^^xsd:gYear ;
bs:hasCafe true .
}
# Then run this to see what it did:
# SELECT ?name ?founded
# WHERE {
# ?shop a bs:Bookshop ;
# bs:locatedIn bt:place-hay-on-wye ;
# rdfs:label ?name .
# OPTIONAL { ?shop bs:founded ?founded }
# }
# ORDER BY ?name
An update produces no result. This is the query that shows what it
did, run against the updated store — and it is what the row
counts below were measured from.
INSERT DATA takes ground triples -- no variables, no WHERE -- and puts them in the store. It is the simplest thing in SPARQL Update and the one you will use least, because most of what you want to add depends on what is already there.
What to take away
INSERT DATA adds ground triples. No variables, no WHERE clause.
An update produces no result, so pair every one with a query that shows what changed. Get into the habit now.
Adding a triple that is already present does nothing. RDF graphs are sets, so updates that only insert are safe to re-run.
INSERT DATA {
bt:shop-foxed-page a bs:Bookshop ; ... .
}
|
'-- ground triples only. A variable here is a syntax error.
before after
------ -----
2 shops in Hay-on-Wye 3
Castle Steps Books Castle Steps Books
The Clock Tower The Clock Tower
The Foxed Page <- new
Pick a subject that does not already exist. INSERT DATA on an
IRI that is already in the store adds to it rather than
replacing it -- which is q119's whole subject, arrived at by
accident.
An update returns nothing: no rows, no count, no graph. Fuseki
answers HTTP 204 and HOLOS prints "inserted 5 deleted 0". That
is the whole feedback, which is why the second half of every
query in this module is a SELECT.
INSERT DATA is also idempotent. Running it twice adds nothing
the second time, because a graph is a set: the same triple is
already there.
EditorHOLOS3Fuseki3bookshop-trail-1.1.ttl
Q117
Removing facts you can name
The Inkwell has closed its cafe. Remove that one fact.
# Q117 Removing facts you can name
# The Inkwell has closed its cafe. Remove that one fact.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
DELETE DATA {
bt:shop-inkwell bs:hasCafe true .
}
# Then run this to see what it did:
# SELECT ?state (COUNT(*) AS ?shops)
# WHERE {
# ?shop a bs:Bookshop .
# OPTIONAL { ?shop bs:hasCafe ?cafe }
# BIND( COALESCE(STR(?cafe), "no statement") AS ?state )
# }
# GROUP BY ?state
# ORDER BY ?state
PREFIXbt:<https://example.org/bookshop-trail/>PREFIXbs:<https://example.org/bookshop-trail/schema#>
DELETE DATA {
bt:shop-inkwellbs:hasCafe true .
}
Then check it worked
An update produces no result. This is the query that shows what it
did, run against the updated store — and it is what the row
counts below were measured from.
SELECT?state (COUNT(*) AS?shops)
WHERE {
?shopabs:Bookshop .
OPTIONAL { ?shopbs:hasCafe?cafe }
BIND( COALESCE(STR(?cafe), "no statement") AS?state )
}
GROUPBY?stateORDERBY?state
How it works
DELETE DATA is the mirror of INSERT DATA: ground triples, removed exactly. It will not accept a variable, which makes it safe and almost useless -- you have to already know the object you are deleting, down to its datatype.
What to take away
DELETE DATA removes ground triples and takes no variables. The terms must match exactly, datatype included.
Deleting a fact leaves the absence of a fact, not its negation. In RDF those are different, and q16's lesson applies here too.
A DELETE that matches nothing is silent. Check with a query rather than assuming.
DELETE DATA { bt:shop-inkwell bs:hasCafe true . }
the triple has to match EXACTLY:
bs:hasCafe true matches
bs:hasCafe "true" does not -- a string
bs:hasCafe "true"^^xsd:boolean matches -- same term
bs:hasCafe ?anything SYNTAX ERROR
+--------------------+--------+-------+
| | before | after |
+--------------------+--------+-------+
| says true | 22 | 21 |
| says false | 11 | 11 |
| says nothing | 0 | 1 |
+--------------------+--------+-------+
That last row is The Inkwell, and it now says NOTHING about a
cafe -- which is not the same as saying it has none. Deleting a
fact leaves silence, not a denial.
Deleting a triple that is not there is not an error. It quietly
does nothing -- so a DELETE DATA that achieves nothing looks
exactly like one that worked.
EditorHOLOS3Fuseki3bookshop-trail-1.1.ttl
Q118
Removing whatever matches
Drop every bs:hasCafe false statement, on the grounds that they say nothing a missing statement would not.
# Q118 Removing whatever matches
# Drop every bs:hasCafe false statement, on the grounds that they say nothing a missing statement would not.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
DELETE WHERE {
?shop bs:hasCafe false .
}
# Then run this to see what it did:
# SELECT ?state (COUNT(*) AS ?shops)
# WHERE {
# ?shop a bs:Bookshop .
# OPTIONAL { ?shop bs:hasCafe ?cafe }
# BIND( COALESCE(STR(?cafe), "no statement") AS ?state )
# }
# GROUP BY ?state
# ORDER BY ?state
PREFIXbs:<https://example.org/bookshop-trail/schema#>
DELETE WHERE {
?shopbs:hasCafe false .
}
Then check it worked
An update produces no result. This is the query that shows what it
did, run against the updated store — and it is what the row
counts below were measured from.
SELECT?state (COUNT(*) AS?shops)
WHERE {
?shopabs:Bookshop .
OPTIONAL { ?shopbs:hasCafe?cafe }
BIND( COALESCE(STR(?cafe), "no statement") AS?state )
}
GROUPBY?stateORDERBY?state
How it works
DELETE WHERE is the shorthand that finally allows variables: the pattern is both the thing to match and the thing to remove, written once. It is the most useful deletion form and the most dangerous, because the pattern that matches too much removes too much.
What to take away
DELETE WHERE matches and deletes with one pattern. It is the form you will reach for most.
Run the pattern as a SELECT first, every time. The pattern is the whole of what makes a deletion safe.
Removing a false statement is not the same as leaving it. Decide which of the two your consumers expect.
DELETE WHERE { ?shop bs:hasCafe false }
-----------------------
matched AND deleted -- the same block does both
equivalent long form:
DELETE { ?shop bs:hasCafe false }
WHERE { ?shop bs:hasCafe false }
+--------------------+--------+-------+
| | before | after |
+--------------------+--------+-------+
| says true | 22 | 22 |
| says false | 11 | 0 |
| says nothing | 0 | 11 |
+--------------------+--------+-------+
Whether this is an improvement depends entirely on whether you
were relying on "false" meaning "we checked, and no". Once the
statement is gone, that shop is indistinguishable from one
nobody has asked about.
Now widen the pattern by one step and read it again:
DELETE WHERE { ?s bs:hasCafe ?o } both kinds gone
DELETE WHERE { ?s ?p ?o } everything gone
q123 is about that second one.
EditorHOLOS2Fuseki2bookshop-trail-1.1.ttl
Q119
Correcting a value
Ex Libris was founded in 1921, not 1919. Change it.
# Q119 Correcting a value
# Ex Libris was founded in 1921, not 1919. Change it.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
DELETE { ?shop bs:founded ?old }
INSERT { ?shop bs:founded "1921"^^xsd:gYear }
WHERE {
?shop a bs:Bookshop ;
bs:founded ?old .
FILTER( ?shop = bt:shop-ex-libris )
}
# Then run this to see what it did:
# SELECT ?name ?founded
# WHERE {
# bt:shop-ex-libris rdfs:label ?name ;
# bs:founded ?founded .
# }
# ORDER BY ?founded
An update produces no result. This is the query that shows what it
did, run against the updated store — and it is what the row
counts below were measured from.
DELETE and INSERT in one operation, sharing a WHERE clause. The WHERE runs once, its bindings feed both templates, and the DELETE half is applied before the INSERT half. Leaving the DELETE out is the commonest mistake in SPARQL Update, and it does not look like a mistake: the new value appears, and so does the old one.
What to take away
DELETE/INSERT/WHERE is the workhorse. One WHERE, two templates, delete applied before insert.
An INSERT with no matching DELETE leaves both values in place. Nothing warns you; the property simply has two.
Bind the old value to a variable in the WHERE and delete it by variable, rather than naming what you think it is.
DELETE { ?shop bs:founded ?old } <- the old triple, by variable
INSERT { ?shop bs:founded "1921" } <- the new one
WHERE { ?shop bs:founded ?old . <- binds ?old, once
FILTER( ?shop = bt:shop-ex-libris ) }
with the DELETE: without it:
bs:founded 1921 bs:founded 1919
bs:founded 1921
---------------
Two founding years, no error, and
a query that asks for one now
returns two rows.
A property being single-valued is a claim your data makes, not
a rule the store enforces. Nothing stops a second value
arriving, which is what owl:FunctionalProperty is for -- it
says what a reasoner may conclude, and still does not stop the
insert.
+----------------------------------------------------------+
| Order inside one operation: DELETE first, then INSERT. |
| So a value can be replaced by one computed from itself. |
+----------------------------------------------------------+
EditorHOLOS1Fuseki1bookshop-trail-1.1.ttl
Q120
Materialising what a path already knows
Every shop is in a country by way of two or three bs:within hops. Write that down as one triple per shop.
# Q120 Materialising what a path already knows
# Every shop is in a country by way of two or three bs:within hops. Write that down as one triple per shop.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
INSERT { ?shop bs:inCountry ?country }
WHERE {
?shop a bs:Bookshop ;
bs:locatedIn ?town .
?town bs:within+ ?country .
?country a bs:Country .
}
# Then run this to see what it did:
# SELECT ?country (COUNT(?shop) AS ?shops)
# WHERE {
# ?shop bs:inCountry ?country .
# ?country rdfs:label ?name .
# FILTER( LANG(?name) = "en" )
# }
# GROUP BY ?country
# ORDER BY DESC(?shops) ?country
An update produces no result. This is the query that shows what it
did, run against the updated store — and it is what the row
counts below were measured from.
INSERT ... WHERE is CONSTRUCT that keeps its output. The WHERE walks the path from module 05; the template records the answer. Afterwards the same question is a single triple pattern, which is faster to answer and easier for anything downstream to consume.
What to take away
INSERT ... WHERE is a CONSTRUCT whose output is kept. Anything you can construct, you can materialise.
Materialised triples are stale the moment the facts they came from change. Decide who re-runs them, and when.
Keep derived triples separable from asserted ones -- a named graph is the cheapest way -- so they can be dropped and rebuilt.
what the path costs, every time it is asked:
?shop bs:locatedIn ?town .
?town bs:within+ ?country . <- two or three hops,
?country a bs:Country . evaluated per shop
what the update leaves behind:
?shop bs:inCountry ?country . <- one hop
+-------------+--------+
| England | 20 |
| Scotland | 9 |
| Wales | 4 |
+-------------+--------+
33 shops, each in exactly one country
This is materialisation, and it is a trade rather than a win:
faster to query the derived triples are now data,
simpler downstream and nothing keeps them true. Move a
shop to another town and bs:inCountry
still says the old country.
So either re-run it after every change, or do not store it and
pay the path cost at query time. What you must not do is store
it and forget which of the two you chose.
bs:inCountry is not in the vocabulary file. Derived shortcuts
usually are not, which is another reason to keep them clearly
separable -- a named graph is the usual answer, and module 08
has the mechanism.
EditorHOLOS3Fuseki3bookshop-trail-1.1.ttl
Q121
The migration, done in place
Turn the 95 StockRecord nodes into RDF 1.2 annotations, and remove the old ones.
An update produces no result. This is the query that shows what it
did, run against the updated store — and it is what the row
counts below were measured from.
q86 built this graph with CONSTRUCT and left you holding it. Here the same transformation is applied to the store, and then a second operation removes what it replaced -- the half CONSTRUCT cannot do. Two operations separated by a semicolon, applied in order.
What to take away
Several operations in one request, separated by ';', run in order, and each sees the effect of the last.
Migrate then delete, in that order. The reverse loses the data the migration needed.
Derive new IRIs from stable existing ones and the whole request becomes safe to re-run. That is worth more than it sounds at 3am.
operation 1 read the old shape, write the new one
operation 2 delete the old shape
INSERT { ... } WHERE { ?record a bs:StockRecord ; ... } ;
DELETE WHERE { ?record a bs:StockRecord ; ?p ?o }
^
the semicolon. Operations run in order, and the
second sees what the first did -- which is why
the delete must come second, and why writing it
first quietly produces nothing at all.
before after
------ -----
95 bs:StockRecord nodes 0
0 bs:stocks triples 95
0 reifiers 95, each with copies and price
Idempotence is what makes this survivable. The reifier IRI is
derived from the record's own name, so a migration interrupted
half way can simply be run again: the triples it already wrote
are written again to no effect.
+----------------------------------------------------------+
| Take a backup first. There is no transaction spanning |
| the two operations on every engine, no undo, and no |
| prompt. `holos backup` and Fuseki's tdb2.tdbbackup are |
| the two this course uses. |
+----------------------------------------------------------+
EditorHOLOS2Fuseki2bookshop-trail-1.1.ttl
Q122
Moving whole graphs about
Copy one named graph, then drop another, without touching a single triple pattern.
# Q122 Moving whole graphs about
# Copy one named graph, then drop another, without touching a single triple pattern.
# Data: bookshop-trail.trig
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
COPY bt:graph-shops TO bt:graph-shops-backup ;
DROP GRAPH bt:graph-events
# Then run this to see what it did:
# SELECT ?graph (COUNT(*) AS ?triples)
# WHERE {
# GRAPH ?graph { ?s ?p ?o }
# }
# GROUP BY ?graph
# ORDER BY ?graph
PREFIXbt:<https://example.org/bookshop-trail/>
COPY bt:graph-shops TO bt:graph-shops-backup ;
DROP GRAPHbt:graph-events
Then check it worked
An update produces no result. This is the query that shows what it
did, run against the updated store — and it is what the row
counts below were measured from.
Graph management is its own small language: LOAD, CLEAR, DROP, COPY, MOVE and ADD work on whole graphs at a time. They are far faster than the equivalent INSERT/DELETE, and they are how a staging graph gets promoted to a live one.
What to take away
COPY, MOVE, ADD, DROP and CLEAR act on whole graphs and are much cheaper than the pattern-based equivalent.
COPY and MOVE empty the destination first; ADD does not. DROP removes the graph, CLEAR only empties it.
LOAD fetches a URL of the requester's choosing. Treat it like SERVICE, and expect a careful engine to refuse it.
COPY <a> TO <b> b is emptied, then a's contents put in it
MOVE <a> TO <b> the same, and then a is dropped
ADD <a> TO <b> a's contents added; b keeps what it had
DROP GRAPH <a> the graph and its name, gone
CLEAR GRAPH <a> emptied, but the name remains
the difference that catches people:
COPY destination emptied first ADD destination kept
the trail dataset, in graphs:
bt:graph-places places, councils, regions, countries
bt:graph-shops the bookshops
bt:graph-people authors and publishers
bt:graph-events readings, launches, fairs
... and six more
after COPY bt:graph-shops TO bt:graph-shops-backup
and DROP GRAPH bt:graph-events :
shops-backup holds exactly what shops holds
events is gone -- not empty, gone
LOAD <http://somewhere/data.ttl> INTO GRAPH <g> fetches over
the network, and it is the same exposure as SERVICE in q108:
a URL chosen by whoever wrote the request, fetched by your
server. HOLOS refuses remote LOAD for that reason.
Note. Fuseki and HOLOS. Comunica's update handling of whole-graph operations over an in-memory store is not something this course relies on, and the editor cannot run an update at all.
EditorHOLOS10Fuseki10bookshop-trail.trig
Q137
Emptying, moving and merging graphs
Empty one graph, move a second into a third, and merge a fourth into it.
# Q137 Emptying, moving and merging graphs
# Empty one graph, move a second into a third, and merge a fourth into it.
# Data: bookshop-trail.trig
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
CLEAR GRAPH bt:graph-claims ;
MOVE bt:graph-trail TO bt:graph-archive ;
ADD bt:graph-people TO bt:graph-books
# Then run this to see what it did:
# SELECT ?graph (COUNT(*) AS ?triples)
# WHERE {
# GRAPH ?graph { ?s ?p ?o }
# }
# GROUP BY ?graph
# ORDER BY ?graph
PREFIXbt:<https://example.org/bookshop-trail/>
CLEAR GRAPHbt:graph-claims ;
MOVE bt:graph-trail TO bt:graph-archive ;
ADD bt:graph-people TO bt:graph-books
Then check it worked
An update produces no result. This is the query that shows what it
did, run against the updated store — and it is what the row
counts below were measured from.
q122 covered COPY and DROP. These are the other three, and the distinctions between them are the whole of the vocabulary: CLEAR empties but keeps the name, MOVE is a copy that removes the source, and ADD merges without emptying the destination first.
What to take away
CLEAR empties a graph and keeps its name; DROP removes both.
MOVE, COPY and ADD differ in two ways: whether the destination is emptied first, and whether the source survives. ADD is the merge.
LOAD fetches a URL of the requester's choosing, which is why a careful engine refuses it or puts an allow-list in front.
CLEAR GRAPH <a> a is empty. The name still exists.
DROP GRAPH <a> a is gone. Asking for it is an error.
MOVE <a> TO <b> b emptied, a's contents moved in, a dropped
COPY <a> TO <b> b emptied, a's contents copied in, a kept
ADD <a> TO <b> b KEPT, a's contents added, a kept
applied to the trail dataset:
CLEAR GRAPH bt:graph-claims 243 triples -> 0, name kept
MOVE bt:graph-trail TO bt:graph-archive
trail gone, archive has 429
ADD bt:graph-people TO bt:graph-books
books 918 + people 301 = 1219
+----------------------------------------------------------+
| ADD is the one to reach for when merging, and the one |
| people reach for COPY instead of. COPY silently empties |
| the destination first, so a merge written with COPY |
| destroys whatever was already there. |
+----------------------------------------------------------+
LOAD <url> INTO GRAPH <g> is the sixth of these, and it
fetches over the network. That makes it the same exposure as
SERVICE (q108): a URL the requester chose, fetched by your
server, from inside your network. HOLOS refuses remote LOAD
for that reason, and an endpoint that allows it should have an
allow-list in front of it.
All six are also SILENT-able: CLEAR SILENT GRAPH <a> succeeds
rather than failing when there is no such graph.
Note. Fuseki and HOLOS, for the same reason as q122. Note that bt:graph-claims survives as a name with nothing in it, so it disappears from a result grouped by GRAPH -- an empty graph has no triples to group.
EditorHOLOS9Fuseki9bookshop-trail.trig
Q139
Loading a file from the web
Fetch a graph over HTTP and put it in a named graph.
# Q139 Loading a file from the web
# Fetch a graph over HTTP and put it in a named graph.
# Data: bookshop-trail.trig
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
LOAD <https://raw.githubusercontent.com/pwin/SPARQL_Course/main/data/04-bookshops.ttl>
INTO GRAPH bt:graph-imported
# Then run this to see what it did:
# SELECT ?graph (COUNT(*) AS ?triples)
# WHERE {
# GRAPH ?graph { ?s ?p ?o }
# }
# GROUP BY ?graph
# ORDER BY ?graph
PREFIXbt:<https://example.org/bookshop-trail/>
LOAD <https://raw.githubusercontent.com/pwin/SPARQL_Course/main/data/04-bookshops.ttl>
INTO GRAPHbt:graph-imported
Then check it worked
An update produces no result. This is the query that shows what it
did, run against the updated store — and it is what the row
counts below were measured from.
LOAD takes a URL, fetches whatever RDF is there, and adds it. It is the fastest way to get a file into a store and the operation most worth being careful about, because the URL comes from whoever wrote the request rather than from whoever runs the server.
What to take away
LOAD fetches a URL and adds the triples, optionally INTO GRAPH. SILENT stops a failed fetch from failing the request.
The URL is chosen by the request, not the operator. Treat an update endpoint as a way to make your server fetch arbitrary addresses.
The Graph Store Protocol is the better route for loading your own files, because the client supplies the bytes.
LOAD <https://raw.githubusercontent.com/.../04-bookshops.ttl>
INTO GRAPH bt:graph-imported
588 triples arrive in a graph that did not exist before.
LOAD <url> into the default graph
LOAD <url> INTO GRAPH <g> into a named one
LOAD SILENT <url> carry on if the fetch fails
+----------------------------------------------------------+
| Same exposure as SERVICE (q108). The server makes an |
| HTTP request to an address a stranger named, from |
| inside your network: |
| |
| LOAD <http://169.254.169.254/latest/meta-data/> |
| |
| HOLOS refuses remote LOAD outright, for that reason. |
| Fuseki allows it, so an open update endpoint is an |
| open fetcher as well as an open writer. |
+----------------------------------------------------------+
The Graph Store Protocol does the same job over plain HTTP
and puts the choice of file on the client rather than in the
query -- see "Talking to an endpoint". For loading your own
data that is the better tool; LOAD is for when the request
itself has to say where the data comes from.
Formats: the fetched document is parsed by its content type,
so a server that serves Turtle as text/plain will produce a
parse error rather than a graph. That is the commonest reason
a LOAD of a working URL fails.
Note. Fuseki only, and it needs the internet. HOLOS refuses remote LOAD for the reason in the diagram; the browser editor cannot run an update at all.
# Q123 The update that empties the store
# What does one careless pattern cost?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
DELETE WHERE {
?s ?p ?o .
}
# Then run this to see what it did:
# SELECT (COUNT(*) AS ?triplesLeft)
# WHERE {
# ?s ?p ?o .
# }
DELETE WHERE {
?s?p?o .
}
Then check it worked
An update produces no result. This is the query that shows what it
did, run against the updated store — and it is what the row
counts below were measured from.
SELECT (COUNT(*) AS?triplesLeft)
WHERE {
?s?p?o .
}
How it works
Everything. DELETE WHERE with three variables matches every triple in the default graph and removes all of them. There is no confirmation, no transaction to roll back on most setups, and no undo. It is worth running once, deliberately, on a copy -- the point lands better than a warning does.
What to take away
DELETE WHERE { ?s ?p ?o } empties the default graph, succeeds quietly, and cannot be undone.
Write every deletion as a SELECT first. It costs one run and it is the only real safeguard.
Separate read and write endpoints, and back up before migrating. An update endpoint open to the internet is an open door.
DELETE WHERE { ?s ?p ?o }
4826 triples -> 0
+----------------------------------------------------------+
| no prompt no confirmation no undo no error |
| the operation succeeds. That is the problem with it. |
+----------------------------------------------------------+
The three habits that prevent it:
1 Write the WHERE as a SELECT first and look at the rows.
Every deletion in this module was written that way.
2 Keep the endpoint read-only unless it needs to write.
Fuseki serves /query and /update separately, and most
deployments should never expose the second one.
3 Back up before a migration, not after noticing.
holos backup --store DIR --to DIR
java -cp ... tdb2.tdbbackup --loc DIR
And a fourth, for the query itself: LIMIT does nothing here.
There is no such thing as deleting the first ten matches.
A note on what "the default graph" means. This removes the
default graph only; named graphs survive, which makes the
damage look smaller than it is until somebody checks. DROP ALL
is the one that takes everything.
EditorHOLOS1Fuseki1bookshop-trail-1.1.ttl
Module 18
Inference
A reasoner derives new triples from the ones you have plus the rules in the vocabulary. For the commonest cases -- transitivity, inverse properties, class hierarchies -- SPARQL does the same job at query time, on every engine, with nothing stored and nothing to keep up to date. This module writes those inferences as queries; the reference section on reasoning measures what happens when you switch a real reasoner on instead, and no two of the three engines agree.
# Q140 The closure a reasoner would give you
# How many containment facts are there once you follow the chain?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
SELECT (COUNT(*) AS ?pairs)
WHERE {
SELECT DISTINCT ?place ?container
WHERE {
?place bs:within+ ?container .
}
}
bs:within is declared owl:TransitiveProperty, which tells a reasoner it may derive York-in-England from York-in-Yorkshire and Yorkshire-in-England. A property path computes the same set at query time. The two answers are identical, and one of them needs no reasoner, no extra storage and nothing to re-run after a write.
What to take away
owl:TransitiveProperty tells a reasoner it may close a chain. p+ computes the same set at query time on any SPARQL 1.1 engine.
RDFS has no transitivity rule, so an RDFS reasoner leaves a transitive property exactly as asserted. Check which profile you have before assuming.
Materialised inference is a cache. Ask who invalidates it before you decide it is cheaper than a path.
asserted after transitivity
York -> Yorkshire York -> Yorkshire
Yorkshire -> England York -> England <- derived
England -> GB York -> GB <- derived
Yorkshire -> England
63 triples Yorkshire -> GB <- derived
England -> GB
...
186 pairs
measured, all four routes:
+--------------------------------+-----------+
| asserted | 63 |
| Jena riotcmd.infer --rdfs | 63 | RDFS has no
| HOLOS holos entail | 63 | transitivity rule
| HyLAR OWL 2 RL (the editor) | 186 |
| Jena OWLMicro via assembler | 186 | two OWL reasoners,
| ?s bs:within+ ?o | 186 | one path, one answer
+--------------------------------+-----------+
The two OWL reasoners and the property path agree exactly.
The two RDFS reasoners cannot do it, because transitivity is
an OWL notion and RDFS has no rule for it.
+----------------------------------------------------------+
| A reasoner writes 186 pairs down and they go stale the |
| moment a place moves. A path recomputes them every time |
| and is never wrong. The reasoner wins when the same |
| closure is queried constantly and the data barely |
| changes; the path wins the rest of the time. |
+----------------------------------------------------------+
Editor1HOLOS1Fuseki1bookshop-trail-1.1.ttl
Q141
Filling in the other direction
Every work has an author; how many author-to-work links are there if you count both properties?
# Q141 Filling in the other direction
# Every work has an author; how many author-to-work links are there if you count both properties?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bs: <https://example.org/bookshop-trail/schema#>
SELECT (COUNT(*) AS ?links)
WHERE {
SELECT DISTINCT ?author ?work
WHERE {
?work ^bs:wrote|bs:author ?author .
}
}
bs:wrote is declared owl:inverseOf bs:author, so a reasoner would make each complete from the other. The data does not have them complete: 74 works name an author, and only 68 authors name a work. Reading the property backwards with ^ recovers all 74 without deriving anything.
What to take away
owl:inverseOf lets a reasoner complete each direction from the other. ^ reads a predicate backwards and needs no reasoner.
Data with an inverse pair is rarely complete in both directions. Ask both ways and compare before trusting either.
UNION of the two directions and ^p|q are the same query. The second is shorter and the engine can plan it as one path.
bs:wrote owl:inverseOf bs:author .
asserted what a reasoner would add
?a bs:wrote ?w 68 6 more bs:wrote
?w bs:author ?a 74 6 more... no: none. 74 already.
+----------------------------+-------+
| ?a bs:wrote ?w | 68 |
| ?w bs:author ?a (flipped) | 74 |
| either of the two | 74 | <- this query
+----------------------------+-------+
So bs:wrote is missing six links that bs:author has. A
reasoner would fix that by deriving them. So does this:
{ ?author bs:wrote ?work } UNION { ?work bs:author ?author }
and so does the shorter form, which is the one to remember:
?work ^bs:wrote|bs:author ?author
^ reads a predicate backwards (module 05, q32). An inverse
declaration in the vocabulary is a promise about what a
reasoner may do; ^ is the same journey with no promise
required.
Worth noticing which way round the gap is. Nothing warns you
that one direction of an inverse pair is less complete than
the other -- the data simply answers differently depending on
which way you ask, and that is a thing to check for rather
than to discover.
Editor1HOLOS1Fuseki1bookshop-trail-1.1.ttl
Q142
What RDFS would actually add here
How deep is the class hierarchy a reasoner would have to walk?
# Q142 What RDFS would actually add here
# How deep is the class hierarchy a reasoner would have to walk?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?subclass ?superclass
WHERE {
?subclass rdfs:subClassOf+ ?superclass .
FILTER( ?subclass != ?superclass )
}
ORDER BY ?superclass ?subclass
RDFS's most useful rule says that something in a class is also in every superclass of it. How much that gives you depends entirely on how deep the hierarchy is and how completely the data is typed. Here: six subclass pairs, and every instance already carries its types. Which is why switching RDFS on adds so little.
What to take away
RDFS earns its keep on deep hierarchies and partially typed data. This dataset has neither, so it adds 140 triples.
rdfs:subClassOf+ walks the hierarchy with no reasoner, and shows you how much one would have to do.
The size of an inference closure is a property of your data, not of the engine. Measure before choosing.
the whole class hierarchy of this vocabulary:
bs:Settlement rdfs:subClassOf bs:Place
bs:CouncilArea rdfs:subClassOf bs:Place
bs:Region rdfs:subClassOf bs:Place
bs:Country rdfs:subClassOf bs:Place
bs:Author rdfs:subClassOf bs:Person
bs:Translation rdfs:subClassOf bs:Work
6 pairs. Nothing is more than one hop deep.
and so, measured:
+--------------------------------+---------+----------+
| | triples | usefully |
| | | new |
+--------------------------------+---------+----------+
| asserted | 4,826 | - |
| Jena riotcmd.infer --rdfs | 4,826 | 0 |
| HOLOS holos entail | 5,051 | 0 |
| HyLAR OWL 2 RL | 8,777 | 3,952 |
+--------------------------------+---------+----------+
Neither RDFS reasoner derives a usable new fact here. HOLOS
writes 225 triples, of which 133 are reflexive -- X is a
subclass of X -- and 92 type things as a class with no name
(q143). Jena writes none at all.
That is not a failing of either. It is what RDFS is for
meeting a dataset that has no deep hierarchy and is already
fully typed. On data where instances carry only their most
specific type and the hierarchy is eight deep, the same
reasoner earns its keep.
rdfs:subClassOf+ walks it without one, and gives the six pairs
directly -- which is also how you check a hierarchy for the
cycle that would make a reasoner loop.
+----------------------------------------------------------+
| Measure the hierarchy before you switch a reasoner on. |
| This query is that measurement, it takes a second, and |
| it will sometimes tell you not to bother. |
+----------------------------------------------------------+
Editor6HOLOS6Fuseki6bookshop-trail-1.1.ttl
Q143
The inference nobody wanted
Which properties would a reasoner use to type things as a class with no name?
# Q143 The inference nobody wanted
# Which properties would a reasoner use to type things as a class with no name?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT ?property (COUNT(DISTINCT ?s) AS ?wouldBeTyped)
WHERE {
?property rdfs:domain ?anonymousClass .
FILTER( isBLANK(?anonymousClass) )
?s ?property ?o .
}
GROUP BY ?property
ORDER BY ?property
rdfs:domain does not constrain anything. It licenses an inference: if ?s has this property then ?s is in this class. When the domain is an anonymous union class -- written [ owl:unionOf ( A B ) ] because the property applies to two kinds of thing -- a reasoner dutifully types every subject as a blank node. True, and unusable.
What to take away
rdfs:domain licenses an inference; it does not constrain. It cannot make anything invalid, only make more things true.
An anonymous union class as a domain makes a reasoner type every subject as a blank node -- correct, and impossible to query.
A closure contains everything the rules permit. Look at what a reasoner derived before deciding to keep it.
the vocabulary says:
bs:locatedIn rdfs:domain [ a owl:Class ;
owl:unionOf ( bs:Bookshop bs:Publisher ) ] .
RDFS rule rdfs2 therefore derives, for all 46 subjects:
bt:shop-inkwell rdf:type _:b0 .
---
a class with no name. You cannot write a
query against it, cannot report it, cannot
link to it. It is 92 of the 225 triples
HOLOS entails.
+------------------+---------------+
| bs:founded | 46 | 33 shops + 13 publishers
| bs:locatedIn | 46 | the same 46
+------------------+---------------+
Both properties apply to a shop or a publisher, so both
declare the same anonymous union as their domain, and both
make the reasoner type the same 46 things as the same
nameless class.
Two lessons, and the second is the bigger one:
1 rdfs:domain is not a constraint. It never says "this is
wrong"; it says "therefore this is also true". A shop
with a bs:locatedIn is not checked against anything --
it is typed. Use SHACL (module 12) when you want the
checking meaning.
2 A reasoner produces everything its rules permit, not
what you were hoping for. Some of that is noise, and
noise you have to store and search past.
Blank nodes as classes are module 16's subject arriving from
an unexpected direction, and q109's census is what finds them.
Editor2HOLOS2Fuseki2bookshop-trail-1.1.ttl
Q144
A reasoner you can read
Materialise the containment closure as a graph, with the derived triples marked as derived.
# Q144 A reasoner you can read
# Materialise the containment closure as a graph, with the derived triples marked as derived.
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX bt: <https://example.org/bookshop-trail/>
PREFIX bs: <https://example.org/bookshop-trail/schema#>
CONSTRUCT {
?place bs:withinTransitive ?container .
}
WHERE {
?place bs:within+ ?container .
}
A CONSTRUCT is a single inference rule you can read, test and change. This one does what an OWL reasoner does for transitivity, and adds the thing a reasoner does not: it says which triples it made up, so they can be told apart afterwards and dropped when they go stale.
What to take away
A CONSTRUCT is one inference rule, written where you can read it. That is most of what a reasoner does, minus the surprises.
Put derived triples on their own predicate or in their own graph. Mixing them with asserted ones destroys provenance and cannot be undone.
q120 turns this into an INSERT. The difference between deriving and storing is who has to remember to re-run it.
the rule, in one query:
?place bs:within+ ?container IF
-> ?place bs:withinTransitive ?container THEN
186 triples out, on a different predicate from the 63 that
were asserted. That separation is deliberate:
bs:within asserted, 63. Untouched.
bs:withinTransitive derived, 186. Droppable.
+----------------------------------------------------------+
| A reasoner that writes into the same predicate leaves |
| you unable to answer "who said this?". Load derived |
| triples into their own graph, or onto their own |
| predicate, or both -- HOLOS uses a graph, this uses a |
| predicate, and either beats mixing them. |
+----------------------------------------------------------+
What CONSTRUCT gives you that a reasoner does not:
you can read the rule it is one query
you can test it q140 counts what it makes
you can change it without changing the data
it stops where you say no unwanted closure (q143)
it runs on every engine no profile to negotiate
What a reasoner gives you that this does not: all the other
rules at once, and consistency you did not have to think
about. Module 17 q120 is this pattern written as an INSERT,
which is materialisation proper.
Editor186HOLOS186Fuseki186bookshop-trail-1.1.ttl
Q145
What entailment cannot do
The vocabulary says a Place and a Bookshop can never be the same thing. What happens if one is?
# Q145 What entailment cannot do
# The vocabulary says a Place and a Bookshop can never be the same thing. What happens if one is?
# Data: bookshop-trail-1.1.ttl
# The Bookshop Trail -- https://github.com/pwin/SPARQL_Course
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>
SELECT ?thing ?classA ?classB
WHERE {
?axiom a owl:AllDisjointClasses ;
owl:members/rdf:rest*/rdf:first ?classA ,
?classB .
FILTER( STR(?classA) < STR(?classB) )
?thing a ?classA , ?classB .
}
ORDER BY ?thing ?classA ?classB
Nothing, as far as any query is concerned. Entailment only ever adds triples; it has no way to remove one or to answer no. A disjointness axiom makes an ontology inconsistent rather than making a query empty, and a rule reasoner will usually carry on regardless. Checking is a separate job with separate tools.
What to take away
Entailment is monotonic. It adds; it cannot retract, contradict or report. No amount of reasoning will tell you your data is wrong.
A disjointness axiom makes the ontology inconsistent, not the query empty. Most rule reasoners will not even mention it.
Validation is a separate tool: SHACL, or a query like this one run as a test. Module 12 does both.
the vocabulary asserts:
[] a owl:AllDisjointClasses ;
owl:members ( bs:Place bs:Bookshop bs:Person ... ) .
if something were both a Place and a Bookshop:
OWL DL the ontology is INCONSISTENT. Everything
follows from it, so every answer is
meaningless -- which a DL reasoner reports
and a rule reasoner mostly does not.
OWL 2 RL derives owl:Nothing or a clash flag, or
simply carries on. Rule reasoners are built
to add, not to object.
your query returns the row. Nothing checked anything.
+----------------------------------------------------------+
| Entailment is monotonic: adding data never withdraws a |
| conclusion. So it can never say "this is wrong", only |
| "and therefore this as well". Every validation question |
| you have is outside it. |
+----------------------------------------------------------+
Which is what SHACL is for, and module 12 uses it in earnest.
This query is the SPARQL version of the same check: find
anything that is in two classes declared disjoint. It returns
nothing on this dataset, and returning nothing is the answer
you want.
disjointness declared, in pairs 15 classes, 2 axioms
things violating it 0
A test that passes silently is worth writing down: q76 is the
ASK form of the same idea, and the reason to prefer it is
that a boolean false is harder to overlook than an empty
table.
Editor0HOLOS0Fuseki0bookshop-trail-1.1.ttl
Reference
Find a feature
Every SPARQL keyword and function this course
demonstrates, and which queries demonstrate it — 111 of
111 in the catalogue. Built by scanning the query bodies, so it
cannot drift from them; the feature name links to the section of the
specification that defines it.
Module 18 derives things with property paths and
CONSTRUCT, which runs anywhere. This is the other route: turning a
reasoner on. All three engines can, none of them agrees with the others,
and the differences are large enough to change what your queries
return.
4,826 asserted triples in, and two probes on the way out: how many
bs:within triples exist afterwards, and how many things
are typed bs:Place.
distinct usefully bs:within a bs:Place
triples new
asserted, no reasoning 4,826 - 63 64
Jena riotcmd.infer --rdfs 4,826 0 63 64
HOLOS holos entail 5,051 0 63 64
HyLAR OWL 2 RL (the editor) 8,777 3,952 186 64
Jena OWLMicro via assembler -- -- 186 64
SPARQL ?s bs:within+ ?o 4,826 0 186 64
---
no reasoner, same answer
Two things to take from that table. Neither RDFS
reasoner derives a single usable new fact on this dataset — the
next block is why. And the last row: a property path computes the same
transitive closure the OWL reasoners do, at query time, on every
engine, with nothing switched on and nothing stored. Module 18 q140 is
that query.
"Usefully new" needs defining, because the raw counts flatter both
RDFS reasoners. Jena emits 9,375 lines that reduce to 4,826 distinct
triples — every one of them already asserted, differing only in
blank node labels. HOLOS writes 225, of which 133 are reflexive
axioms (X rdfs:subClassOf X, p rdfs:subPropertyOf
p) and the remaining 92 type things as a class with no name.
True, all of it, and not a fact you can query for.
Why RDFS adds so little here
Zero is not a bug in either engine. This dataset gives RDFS nothing
to do: the class hierarchy is six subclass pairs in total, none more
than one hop deep, and every instance already carries its types
explicitly. rdfs9, the rule that does most of RDFS's
work, has nothing left to derive. On a dataset where instances carry
only their most specific type and the hierarchy is eight deep, the
same reasoner earns its keep.
What RDFS cannot do at any depth is the interesting part:
bs:within is declared owl:TransitiveProperty
and RDFS has no rule for transitivity, so both RDFS reasoners leave it
at 63. Only the two OWL reasoners close it, and they agree exactly:
186.
The browser editor — HyLAR, OWL 2 RL
Press Show Facts. HyLAR runs in the page, over the graph
currently loaded, and adds what it derives to the view. It is the only
one of the three that reasons where you can watch it happen, which
makes it the right one to learn on.
load data/bookshop-trail-1.1.ttl
press Show Facts
3,952 new triples, about 1.7 seconds
bs:within closes from 63 to 186
bs:author and bs:wrote fill each other in, both ways
OWL 2 RL is a rule language, so it derives new triples and never
contradicts an existing one. Note what that means: it will happily
derive facts from a self-contradictory ontology rather than telling
you the ontology is broken. Consistency checking is a different job,
and SHACL — module 12 — is the tool this course uses for
it.
The design decision worth copying: the derived triples go into their
own named graph. A query sees them only if it asks for that graph, and
DROP GRAPH <https://holos.dev/ns#entailed> undoes
the whole thing exactly. Materialised inference goes stale the moment
the data changes, and being able to throw it away cleanly is what
makes it safe to have.
--entail-budget caps the closure — a reasoner on
the wrong ontology can derive a great deal, and a budget turns a
runaway into an error rather than a full disk.
A finding worth reading twice. 92 of those 225
triples type every shop and publisher as a blank node: the
anonymous owl:unionOf class that bs:locatedIn
declares as its domain. Correct RDFS, and useless — you cannot
write a query against a class with no name. Module 18 q143 finds them
before a reasoner ever runs.
Jena — two routes, very different
The streaming inferencer is one command and does RDFS only:
It streams, so it does not deduplicate. That
command emits 9,375 lines for 4,826 distinct triples — the same
triple derived by several rules is printed once per rule. Sort and
unique before counting anything, or you will report an inference
closure roughly twice its real size and conclude the reasoner did
something.
For anything past RDFS, describe the model with an assembler and
point arq.sparql or Fuseki at it:
That description ships as scripts/owl-micro.ttl —
edit the one file path in it and it runs. It is what produced the 186
in the table. Jena offers RDFS,
OWLMicro, OWLMini and OWLFB in increasing order of what they derive
and decreasing order of how fast they do it, plus a rule language of
its own if none of them fits. Reasoning is applied to the model, so
every query against that dataset sees the derived triples without
asking — convenient, and easy to forget you switched on.
What none of them does
SPARQL has a specification for query-time entailment —
Entailment
Regimes — where the engine answers as though the entailed
triples existed, without ever storing them, and advertises which
regime it implements through its
service
description. None of these three does that. All three
materialise: they compute the new triples and put them somewhere.
what the specification describes what these engines do
query-time entailment materialisation
nothing stored triples written
always current stale as soon as data changes
advertised in the service you have to know
description
So the practical question is never "does it
reason". It is: what does it derive, where does it put it, who
re-runs it after a write, and can you tell derived triples from
asserted ones afterwards? HOLOS answers the last one by construction.
With the other two, a named graph and a discipline are on you.
Reference
Talking to an endpoint
A SPARQL endpoint is an HTTP service, and the
protocol is small enough to learn in one sitting. Once you know it, any
language with an HTTP client can run these queries — no library
required. The examples below assume the Fuseki that
setup-fuseki.ps1 starts.
Fuseki serves a dataset at three paths, and they are deliberately
separate: the one that can change your data is the one you do not
expose.
http://localhost:3030/bookshop/sparql read SELECT, ASK, CONSTRUCT, DESCRIBE
http://localhost:3030/bookshop/update write INSERT, DELETE, LOAD, DROP (module 17)
http://localhost:3030/bookshop/data graphs whole graphs, by PUT / GET / DELETE
The update endpoint is off by default. Start the
server with -Writable to open it, and do that only on a
machine you control. q123 is what an open one costs.
A query is one parameter
GET with the query in the URL is the simplest form, and it is capped
by whatever the server allows in a request line — a few thousand
characters. POST with
application/x-www-form-urlencoded has no such limit and
is what a client should do by default.
# GET -- fine for short queries, and cacheable
curl -G 'http://localhost:3030/bookshop/sparql' \
--data-urlencode 'query=SELECT (COUNT(*) AS ?n) WHERE { ?s ?p ?o }'
# POST -- what to use for anything real
curl -X POST 'http://localhost:3030/bookshop/sparql' \
-H 'Content-Type: application/sparql-query' \
--data-binary @queries/05-property-paths/q31-walking-the-trail-in-either-direction.rq
The second form sends the query as the request body with content type
application/sparql-query, which saves the URL encoding
entirely. Both are in the specification; servers accept both.
Asking for the format you want
The Accept header decides what comes back. This is the
part most people discover by accident, and it is the part that makes
SPARQL pleasant to use from a shell.
# JSON -- the default for SELECT, and what most clients parse
curl -G 'http://localhost:3030/bookshop/sparql' -H 'Accept: application/sparql-results+json' \
--data-urlencode 'query=SELECT * WHERE { ?s ?p ?o } LIMIT 5'
# CSV -- straight into a spreadsheet, or into awk
curl -G 'http://localhost:3030/bookshop/sparql' -H 'Accept: text/csv' \
--data-urlencode 'query=SELECT ?s WHERE { ?s a <https://example.org/bookshop-trail/schema#Bookshop> }'
# Turtle -- for CONSTRUCT and DESCRIBE, which return a graph
curl -G 'http://localhost:3030/bookshop/sparql' -H 'Accept: text/turtle' \
--data-urlencode 'query=DESCRIBE <https://example.org/bookshop-trail/shop-inkwell>'
Ask for TSV rather than CSV when a value might
contain a comma or a newline. TSV escapes them; CSV quotes them, and
quoting survives fewer tools than it should.
Choosing the dataset from outside the query
default-graph-uri and named-graph-uri do
what FROM and FROM NAMED do (q124, q125),
except from the request rather than the query text. Useful when the
query is fixed and the scope is not.
A query that also carries FROM overrides these —
the specification says the protocol parameters are used only when the
query has no dataset clause of its own.
Updates, and PowerShell
An update goes to the write endpoint, by POST only, and returns no
body. Windows has Invoke-RestMethod, which is friendlier
than curl for this and is what the rest of this course uses.
# read
$q = 'SELECT (COUNT(*) AS ?n) WHERE { ?s ?p ?o }'
Invoke-RestMethod -Uri 'http://localhost:3030/bookshop/sparql' -Method Post -Body @{ query = $q } |
ForEach-Object { $_.results.bindings.n.value }
# write -- needs the server started with -Writable
$u = 'INSERT DATA { <urn:a> <urn:b> <urn:c> }'
Invoke-RestMethod -Uri 'http://localhost:3030/bookshop/update' -Method Post -Body @{ update = $u }
No body comes back from an update. A 200 or 204
means it was applied; anything you want to know beyond that needs a
query, which is why every lesson in module 17 has two halves.
Whole graphs, without SPARQL
The Graph Store Protocol treats each named graph as a document:
GET it, PUT to replace it,
POST to merge into it, DELETE to remove it.
For loading a file this is simpler and much faster than an
INSERT.
# replace one graph with a file
curl -X PUT 'http://localhost:3030/bookshop/data?graph=https://example.org/bookshop-trail/graph-shops' \
-H 'Content-Type: text/turtle' \
--data-binary @data/04-bookshops.ttl
# read it back
curl -H 'Accept: text/turtle' \
'http://localhost:3030/bookshop/data?graph=https://example.org/bookshop-trail/graph-shops'
# the default graph is named with ?default rather than ?graph=...
curl -X POST 'http://localhost:3030/bookshop/data?default' \
-H 'Content-Type: text/turtle' --data-binary @data/03-places.ttl
What goes wrong
400 the query did not parse. The body says where.
404 wrong path -- /sparql, not /query, on this server
406 the Accept header asked for something not offered
413 the query was too long for a GET. Use POST.
500 the query parsed and then failed. Often a function the
server does not have -- module 15, and the reason a
column comes back empty rather than erroring.
no response at all
a query with no LIMIT against a large store. Add one,
then read module 13.
Send the same query to two engines when a result
surprises you. That is the whole method this course was built with,
and over HTTP it costs one changed URL.
Reference
The standards
Every module links to the sections it is defined
by; this is the whole reading list in one place. When an engine and this
course disagree, the specification is the thing that settles it —
and the sections are shorter than their reputation suggests.
The query language
SPARQL 1.2 Query LanguageThe one to bookmark. Everything in modules 01 to 09 and 11 to 16 is defined here, and section 17.4 is the function reference you will open most often.
SPARQL 1.2 UpdateINSERT, DELETE and LOAD. The course reads rather than writes, so this appears only in module 12.
SPARQL 1.1 Query LanguageThe previous edition, still what most engines implement in full. Worth having open beside the 1.2 document when an engine disagrees with you.
The data model and its syntaxes
RDF 1.2 Concepts and Abstract SyntaxWhat a triple, a literal, a blank node and a triple term actually are. Appendix B is the one on replacing blank nodes with IRIs.
RDF 1.2 TurtleThe syntax every data file in this course is written in.
RDF 1.2 TriGTurtle plus named graphs, which is what bookshop-trail.trig uses.
RDF 1.2 N-TriplesOne triple per line, no abbreviations. The format to fall back on when a parser disagrees with you about Turtle.
RDF 1.2 Schemardfs:label, rdfs:subClassOf, rdfs:domain and rdfs:range.
RDF 1.2 SemanticsWhat entailment means. Only needed if you start asking what a reasoner is allowed to conclude.
Vocabularies the dataset uses
OWL 2 Structural SpecificationThe normative one. The vocabulary file is OWL 2 DL, and this is the document that says what that requires.
