Well, so fortunately the changeover time here is pretty brief because I'm just jumping straight
into the next presentation.
I think if there are, this presentation touches on a few of the points that we've been talking
about just before now too, so as I finish this up, if there are questions either related
to this talk or if you want to go back and ask more about the previous talk, I think
that's totally fine.
But the, so as is probably clear, the kind of problem that this IMLS planning grant is
intended to address is that the majority of Fedora installations out there are basically
unsupported, so running version 3.x or potentially earlier, there are some 2.x installations
out there too.
And acknowledging that there are many risks to running unsupported software and, you know,
I think those are fairly well known, you know, security risks and just not being able
to support, you know, not having the resources to support the software anymore, and also
recognizing that migrations to Fedora 4 and later have been challenging for a variety
of reasons.
So that's kind of what this grant is targeted at, is just trying to understand what are
some of the things that are making it difficult for the community to move forward and then
what can we do to help that.
So that's really what we're trying to do here.
So this was awarded by the IMLS in the United States and is a planning grant that is focused
on not producing software, but producing an assessment and recommendations.
So that's the idea here.
And we're actually quite far along in the grant process, so we started in October, and
at this point we, well, I'll talk about what's next, but we very nearly completed the work
of the grant.
We also anticipate over this summer to be able to generate this final report and distribute
it along with a set of recommendations.
So in terms of the deliverables, there's kind of an investigation component and an evaluation
component.
In terms of investigation, we conducted a fairly deep environmental scan of kind of
the literature on migrations, not isolated just to Fedora, but just what is kind of the
current state of the art and the lay of the land in terms of migrations as they relate
to repositories in particular.
And then we developed some institutional profiles, so this was quite interesting.
We worked with eight institutions that are running Fedora 3 on a variety of front ends,
Sambera, Islandora, and some custom, and profiled those institutions both in terms of what's
the front end software look like and also what are their data models look like, trying
to get some sense of how difficult or challenging it might be to migrate these into a more current
version of Fedora.
And then we did a review of the current state of migration tools, because there are some
tools that are out there, but they're largely out of date at this point.
Reviewing the Fedora API and also reviewing the Oxford Common File layout with an eye
towards how might any of these things help or hinder migrations.
So I have a link at the end here, but if you go to the Wiki or you just look up the designing
a migration path, you'll find all of that stuff is done, and you can find it on the
Wiki if you want to take a look at it.
The next thing we did was run a community survey, which is closed, that's the bulk of
this presentation is talking about the results of that survey, and producing some results
and analysis off of that.
Just maybe by show of hands, how many of you actually filled out the survey?
Okay.
So not all of you, but nice that we had some participation in that.
Usually the results will still be useful, and this is maybe related to some of the
stuff that Andrew was saying about our desire to work with some pilot institutions as we
kind of move forward, just making sure that your needs are being addressed if you're kind
of still on Fedora 3 and trying to migrate.
So yeah, in terms of what we're hoping to achieve here, try to identify what are the
most common barriers, figure out where we can focus our efforts, and do some combination
of kind of operational and new grant funding opportunities to pursue the work.
So grants tend to take a while to, you know, apply and then get awarded and then start,
although some of them are shorter term.
So some of these efforts will probably take up right away, others will probably pursue
some grant funding to actually do whatever the recommendations suggest.
So I'm going to give just kind of a brief overview of the results so far.
This is all going to be in the final report, but just to give you some sense of what we
found out in the survey.
So we've got 111 responses, which is pretty good.
That was right around what we were targeting.
As is probably expected, the majority of those respondents were from academic institutions,
so 80% or more.
And the majority in the United States, but this also makes sense because the grant is
targeted at the United States just based on where it's funded and the rules of the funding,
although we're happy to get at least some responses from elsewhere in the world.
Most of the respondents actually were developers or software engineers, so that kind of informs
the kind of results we got from the survey, but a number of people were also self-identifying
as repository managers, librarians, or sysadmin, DevOps type people.
But just to give you a sense of who was kind of responding to the survey.
And actually most people reported they had a great deal of influence over the decision
to upgrade or migrate to a new version of the software.
So that's good for us to know, just that most of the people that are responding really
do have a lot of influence at their institutions.
So their opinions here matter.
If there are things that are barriers, then those are real barriers that are preventing
their institution from moving forward.
Perhaps unsurprisingly, most of the repositories held cultural heritage and special collections
data, so the overwhelming majority, but lots still with archival materials and smaller
majorities with scholarly publications and research data.
This was fairly interesting, just a snapshot of the community.
We were kind of asking roughly how many objects, and people define object differently, but
how many objects are in your repository?
So the majority of respondents have 100,000 objects or less in their repository.
So I think it's important just for considering requirements around performance and scale.
If you have 100,000 objects, your requirements are probably quite different than if you were
one of the repositories, the two respondents that have over 10 million objects in their
repository.
So those are vastly different cases in terms of what do you need in order to support your
repository.
And then there's sort of a mix in here, but if you even just combine, you know, the first
two responses here, the vast majority of the respondents have less than half a million
objects in their repositories.
And most people have, you know, one to three full-time equivalent staff that are working
on their repository with a small, you know, fifth or so of respondents actually have quite
a few, so four to six staff, but, you know, close to 12% have less than one FTE working
on the repository.
So that certainly probably informs the results around what are the things that are preventing
you from migrating.
Because if you have less than one FTE working on the software, that's going to make it quite
difficult to undertake any kind of migration.
We asked a question around when the last major upgrade was, and this question ended up being
a little bit confusing.
A lot of these answers, a lot of the sort of write-in answers had to do with, you know,
what do you mean by major upgrade?
And, you know, so we're going to conduct some follow-up interviews here.
But there's quite a mix of results between people that have upgraded within the last
three to five years, say, versus more than five years ago, and 22% have never undertaken
an upgrade at all.
So this was really kind of a pretty diverse, you know, grouping here of lots of different
kind of timescales and when people upgraded.
In terms of how often you should upgrade, this was also a contentious question.
I think something that, you know, should here is fairly interpretive, and that the kind
of long tail of answers there are all the write-in responses of people mostly complaining
about the question, which I think makes sense, you know.
I think it's a difficult question to answer, but quite clearly, something in the range
of two to seven years seems to be like the vast majority of respondents think that's
a reasonable timescale, with some sliver here saying never.
So perhaps, you know, your repository is just frozen in time.
In terms of issues impacting major upgrades, the vast majority here indicated availability
of resources of staffing and funding, that's quite likely related to the question of, you
know, how many people do you have working on your repository, but also the maturity
of the software.
So, you know, libraries tend not to be huge risk takers and are looking to adopt something
once, you know, everyone else has adopted it.
And so a lot of people are looking for something that's proven and mature.
But architecture and design is important, too, and so, you know, over a third of respondents
really indicated that if the new software doesn't have the right architecture or design,
they're probably not overly inclined to migrate.
So this is just, you know, indicating, like, the vast majority of respondents are running
Fedora 3, so 81 out of 102 people that answered this question running Fedora 3, so that's
not totally representative of the community, I would think, because the survey was also
skewed towards migrations, so quite likely people that were already running Fedora 4
or 5 may have just elected not to take the survey, because it was less relevant, but
I think this definitely tells us that the majority of the community, some majority is
still running Fedora 3, so I think we know that.
Yeah?
Sure.
That's quite an interesting curve there, so if we put Fedora 6 on that curve and look
at this next year, we're going to see, what, half a percent of people upgrading to 6?
Well, yeah, I mean, 6 doesn't exist yet, so that's...
No, but my point is really about the trend, so how are you going to address it?
This looks like quite a worrying trend, don't you?
Yeah, well, it's the reason for the grant, right?
I mean, I think we kind of already knew this, but now we're kind of looking at, well, how
do we address this, right, because it's quite clearly a problem.
As Andrew was mentioning, we do want the majority of the community on a recent version of the
software, and so we'll talk about this, of course, but it's important for us to make
sure that Fedora 6 makes migrations easier in a lot of ways, and there's also some good
news from the Islandora community, where there are some 300 or nearly 300 of the institutions
currently running Fedora are running it with Islandora, and that software just released
its new version that works with Fedora 5, so, and they have migration tools to support
that move, so that, I think, will start to move the needle on some of this, but yeah,
you're quite right, it is a problem.
Mark?
I was going to ask a related question, do you know who will be the one who talks about
the need to know how Fedora is front-end?
I don't right now, but we can get some of that from the data, because there were some
responses that folks were indicating what they were running, but I don't know right
now.
Yeah, I'm particularly interested, I guess, in the Fedora 4s, how many of those were independent
local front-ends, and how many were one of those?
Yeah, and so hopefully when we put together the final report, we'll have more information
on that as we dive deeper into the results, but yeah, I think it's a good question trying
to understand, you know, like, to the extent that the community is on different versions,
what front-ends are related to those versions, yeah.
So we've asked a question around, you know, what are your favourite Fedora 3 features,
these are just the top three that people responded to, so, you know, flexible content models,
I mean, it's the F in Fedora, I think that's a big reason people use the software.
The application capabilities, however people chose to define that were highlighted as important,
but also the file system layout, so almost a third of respondents said, you know, their
top feature of Fedora 3 is the file system layout, and quite clearly that's something
that is different in Fedora 4 and 5, but returning much more in that direction in Fedora 6, so
I think that's good data for us to have, that is a barrier for folks that we're currently
on the path to addressing.
And arguably Fedora 4, 5 and 6 do have, you know, support for flexible content models
and preservation capabilities, but it's important to unpack what those mean to people as well.
In terms of favourite Fedora 4 features, this one may be surprising depending on sort of
what you think of Fedora 4 or 5, I know we hear a lot in the community around people
having difficulty with some of the linked data RDF capabilities and really preferring
something more simple and not focusing so much on linked data, but, you know, the majority
of the respondents indicated that linked open data support was their favourite feature of
Fedora 4 or 5.
Now there's a lot to unpack there in terms of what does that mean and are you actually
using it or would you use those capabilities, but quite clearly that's a feature that folks
in the community think is a real benefit.
Also with external component integrations and existing standards that those were popular
as well, but also a good chunk of people responding to the survey weren't sure and some of them
were saying, you know, I'm using Fedora 3, I haven't really started using Fedora 4, I'm
not really sure what the features are yet, but that was an interesting, you know, response
set.
And of course we asked about the barriers to migrating to a supported version of Fedora
and these answers likely not surprising, so insufficient time staff resources, lack
of compatibility with front-end applications, so that, and this would be related to your
question mark, you know, how many of those people are Islandora users, for example, and
they just don't have a path yet and now do, so how much is that being addressed, but also
anyone running a custom front-end would need to rewrite their front-end to work with the
more recent version and that would be a barrier as well.
The change in metadata standards for description, I think here a lot of this is kind of, you
know, the choice whether or not to convert to RDF based metadata and that being a barrier,
and the issues with performance and scale, and I think we're aware of some issues that
are being addressed with Fedora 6 in those categories.
Finally things that will help, I mean, yeah, lots of people want migration tools, we need
to unpack what those tools look like, as Andrew mentioned, we're not going to have a one-click
sort of just convert your repository and you're done type tool, but clearly better
tools would help.
Related metadata migration tools I think would be important to have, and there's been other
work in the community and certainly the Bridge to Haiku folks have worked on some metadata
migration tooling that could be useful here, and also just documentation, so this all helps
inform what we do next and what kind of recommendations we make in terms of what people want.
We ask a question around whether or not you're assessing alternatives to Fedora, and it's
kind of split, but lots of people are, and there's lots of write-in answers here that
will kind of extrapolate on in the final report, mostly just people kind of giving reasons
for why they're exploring alternatives.
This is one of my favorite charts when asking, like, what else are you looking at?
It's basically a rainbow, I mean, there's just, there's lots, and lots of these are
write-in answers, too.
The big one here is dSpace, so that's what most people are looking at, but there's a
long tail of other things out there that people are looking at, which is quite interesting.
This is how the question is phrased, so it's both, it's have you or are you, yeah.
Yes, that's fair.
This is sort of more qualitative analysis, but just pulling out kind of from the write-in
answers, because we just allow people to tell us why they were assessing.
So lots of people are assessing it, because, not surprisingly, they don't feel like the
current platform meets the needs, which is why Fedora 6 is very important for us to get
right.
Also lots of people, Fedora 4 was a big change, and it's a different piece of software, and
that's been a barrier, and long-term support concerns were sort of the third, but that's
fairly far down the list, but I think the notion that Fedora 4 and 5 is not meeting
the needs of some number of community members is a major impetus behind, you know, doing
Fedora 6 and working with pilots and getting input and making sure that whatever we produce
does better meet the needs of the community.
In terms of have you decided to migrate the majority of respondents, and this is 54 of
the total response set were undecided, so that is a risk and an opportunity for us.
I think we are really working hard to deliver Fedora 6 as a platform that meets the needs
and that supports migrations, so that that number of, you know, have you decided to migrate
away ideally goes towards the no category and not the yes category, but there's a lot
of folks out there that are, yeah, they're just not sure.
So I'll just wrap up by saying next steps, we still have to complete the kind of analysis
of the survey results, this is very high level.
We have already scheduled a follow-up kind of group interview with respondents, a subset
of the respondents, so we'll be talking to them next week and asking some more in-depth
questions, clarifying questions, then finalizing the recommendations and publishing the report
would be next, and what's not on here, but related to what we were talking about earlier,
we are looking for institutions that are willing to work with us as pilots as we develop
Fedora 6, and that's not necessarily contributing developer effort, but a lot of it is contributing
sample data, contributing use cases, and just kind of following our efforts and validating
along the way, you know, is this the right direction, are we building this in the right
way, do these tools support your needs, et cetera.
So that's kind of a call-out, you know, if you're at an institution that's currently
on Fedora 3, you haven't migrated to more recent version yet, but you'd like to, but
you're waiting for something, there's something in the way, we'd really like to hear from
you so that we can work with you and make sure that, you know, we can all kind of come
forward together.
So there's more information there, I'd encourage you to look up the Designing a Migration Path
link, all of the stuff that we've done is there, we'll be publishing the report and
survey results over the summer.