fcrepo-message-consumer
- Anna Headley (Unlicensed)
jcoyne
cam156: did you set up fcrepo-message-consumer to use the FileSerializer on your prod install?
11:18
cam156
jcoyne: I have no idea… awed hector^
11:19
cam156
awead^^
11:19
awead
jcoyne: I haven't heard of that, so I'm guessing no
11:19
hector
cam156 jcoyne: I have no idea either.
11:20
jcoyne
I don't think it's totally obvious that that's the preservation strategy for Fedora 4.
11:20
jcoyne
Needs better instructions
11:20
jcoyne
escowles: Is that about right?
11:21
escowles
jcoyne: yes, that's about right -- things are moving towards using camel for that kind of thing now though, but i don't think it's gotten as far as the message-consumer yet
11:27
jonibar has left IRC (Quit: Leaving.)
11:28
awoods has joined (~awoods@c-67-165-245-76.hsd1.co.comcast.net)
11:29
jonibar has joined (~Adium@pool-71-162-33-59.altnpa.east.verizon.net)
11:29
mejackreed has left IRC (Quit: Leaving.)
11:30
jonibar has left IRC (Client Quit)
11:43
pgwillia has joined (~chatzilla@129.128.46.157)
11:49
mejackreed has joined (~mejackree@50.141.67.0)
11:52
jonibar has joined (~Adium@h34.184.101.208.static.ip.windstream.net)
11:53
umgrosscol has joined (~gcol@grosscol.umdl.umich.edu)
11:55
mejackreed has left IRC (Ping timeout: 250 seconds)
12:04
hortongn has left IRC (Ping timeout: 255 seconds)
12:08
mjgiarlo
+1, not totally obviou.
12:08
mjgiarlo
s
12:10
cam156
jcoyne: escowles: Is this something that goes in a config file?
12:10
cam156
fcrepo-message-consumer?
12:10
jcoyne
cam156: it's a totally separate java app
12:11
jcoyne
cam156: https://github.com/fcrepo4/fcrepo-message-consumer#running-the-indexer
12:11
cam156
jcoyne: That should be running on the servers?
12:11
escowles
cam156: jcoyne: yep, it's a separate app that listens for messages coming from fcrepo4 -- it can be on a separate server or the same one
12:12
escowles
cam156: there are modules for indexing updated records in solr, syncing them to a triplestore, or saving them to disk
12:13
escowles
the advantage (over nightly backups) is you get a message every time a record is updated, so you can backup multiple versions throughout the day, and capture changes that would have been overwritten
12:15
cam156
escowles: Is the indexer a necessary item? This is the first I am hearing of it
12:15
escowles
cam156: no, it's not necessary -- you only need it if you want to do something every time records change
12:16
escowles
cam156: in theory a lot of the resque jobs could be done in the fcrepo-message-consumer or fcrepo-camel
12:16
cam156
escowles: I think we are leaning on the old hydra way of index after save at the moment
12:17
awead
cam156 escowles: yeah, we'd want to investigate that as a possible replacement for reque, but not now... :)
12:17
mjgiarlo
what's the current recommendation for folks who care about having filesystems with both binaries and serialized metadata written to disk?
12:18
mjgiarlo
fcr4 already does binaries, it seems, but it'd be great to have "whole objects" on disk at some point.
12:18
awead
mjgiarlo: are you talking about things at the modeshape level?
12:19
mjgiarlo
I'm not sure about levels.
12:20
mjgiarlo
when I have fcr4 running, binaries I upload get spooled off to disk. properties do not, apparently. this is a thing we had in fcr3 and that a bunch of us would like to see in some form.
12:20
escowles
mjgiarlo: awead: the metadata is written to disk in a binary format (modeshape using infinispan store) - if you want to write all the metadata to disk in a human-readable form, then either use message-consumer/camel to do it in real time, or write a tool to crawl the rest api and download everything
12:20
mjgiarlo
ok, so it's the message-consumer again. hence what jcoyne was talking about re: preservation strategy.
12:20
escowles
mjgiarlo: yep, he's referring to this from the tech working group: https://wiki.duraspace.org/display/FF/Outcomes+-+Preservation+Worthiness
12:21
awead
escowles: so where does F4 store metadata, the triples? To a different disk?
12:21
mjgiarlo
ok, good to know. thx
12:21
escowles
awead: f4 has one store for binaries and a separate store for properties
12:21
escowles
binaries are written to disk as-is, named after their sha1 checksums
12:22
awead
escowles: huh. I thought modeshape kept metadata in nodes together with the files.
12:22
escowles
awead: that's what the API looks like (binaries are just another property in the JCR/Modeshape API)
12:23
escowles
awead: but when they're written to disk, they use two separate stores
12:23
mjgiarlo
escowles++
12:23
awead
escowles++ # thanks