Numerous
steps have been taken to ensure the reliability of the data collected with the
PLAISIR©
system. A certain number
have already been described, yet, given the importance of the subject, we will
cover them all in this section.
The
key of the evaluation process is of course the nurse-evaluator.
This person generally belongs to the institution under consideration
and a priori, has never completed any evaluations with a format or procedures
similar to those of the PLAISIR©
system.
She must then be trained to perform the PLAISIR©
evaluations.
This training session has two stages.
The first consists of a group
training session
(maximum
of twelve participants with groups of four or five generally) lasting
two
to three days.
This session is essentially a review of the FRAN©, heading by heading.
The reference manual of the system supports this whole process.
The theoretical approach is followed by a practical one in which the "students"
evaluate a prototype client using the FRAN©.
The
second step of the process consist of an
on-site
individual training session
which
lasts two days.
This training session takes the following form.
Following
the 2 to 3 day, "group" training session, the "student"-evaluator
returns to her institution and completes 10 PLAISIR©
evaluations.
An instructor will then meet her in her institution.
The instructor randomly chooses 3 clients among the 10 evaluated,
independently revaluates them and finally compares her evaluations with the
student's evaluations.
Finally, she "reviews" with the student the seven
remaining evaluations.
This process lasts a day.
The "student" is then asked to perform ten other PLAISIR©
evaluations.
Once these are done, the instructor returns to the institution,
randomly chooses 6 clients among the ten just evaluated, independently
assesses them and finally compares her evaluations with those of the student.
This process lasts a day.
The
process of desk-reviewing all the evaluations allows - as a byproduct - the
identification of "problem" evaluators to which one offers a "follow-up
training", in groups or individually.
The format of these follow-ups is variable.
It depends on the types of problems that are detected.
There are also "refresher sessions" which are aimed at
trained evaluators who have not performed evaluations for a prolonged period
(a year or more).
The
number of evaluators is purposely limited.
At the system's inception in 1983, one had a target of training one
evaluator per 100 beds to ensure that each one performs a sufficient number of
evaluations per year to maintain her skill.
With time, one has trained an average of one evaluator for
approximatively fifty beds. This number, in most cases, allows for a good
quality of data collection.
The training of a larger number of evaluators than planned is due to
the small size of a number of institutions and to the fact that, in certain
facilities, management wanted the head-nurses of the respective units to be
responsible of the evaluations in their unit.
One
of the key elements of data quality assurance is the desk-review of all the
evaluations and
their eventual "correction" following the telephone call to
the evaluator/care-givers of the client.
To detect the problems, the nurse-desk-reviewers is aided by the
computer's program which detects certain anomalies and signals them to the
desk-reviewers in the IMPFRANs©. The
dual character of the PLAISIR©
evaluations is of great importance in the
desk-reviewing process.
In fact, the FRAN© allows one to collect two portraits of the same
client : his
bio-psycho-social profile and his profile of nursing care and assistance
services required.
An experienced desk-reviewer rapidly detects inconsistencies between
these two portraits which allows her to prepare the questions she will ask
during the telephone call to the evaluator or care-giver.
It is also important to note that when the IMPFRAN© of a client is
reviewed, it is always compared with the previous IMPFRAN© if the client had
already been evaluated with the PLAISIR©
system.
This comparison is helpful in detecting incongruencies.
Finally, one must note that experience is important in such a system of
data verification.
Certain desk-reviewers have reviewed since 1983 several tens of
thousands IMPFRANs©, what helps them considerably in their task.
The
process of desk-reviewing and correction is also an ongoing training
experience for the evaluators as they are informed of their errors and
omissions during the telephone contact with the reviewers.
Furthermore, the nurses responsible of the desk-reviewing are also
responsible of the training of evaluators.
They are fully aware of the FRAN© items that cause the most difficulties
to the evaluators and can focus on these items in the training-sessions.
Finally, these same nurse-desk-reviewers pilot the revision/updating
process of the PLAISIR©
system (reference manual and FRAN© form).
In
addition to the (microscopic) desk-review
of the IMPFRANs©, there is the (macroscopic) review
of the PLAISIR©
system outputs.
This last step allows for the eventual detection of systematic bias
(inflated evaluations, for example) which may have gone unnoticed in the
desk-review of individual evaluations.
During the process of reviewing the outputs, one always compares the
current outputs of a program or institution with its last outputs.
Other
important elements of the quality assurance of data are the standards of care
which were mentioned earlier and which are a necessary condition for the
homogeneousness of evaluation and the reference manual which gives accurate
and detailed definitions of each of the FRAN© headings.
Finally,
in very rare cases, where all these measures seem insufficient to ensure
reliability of the data, an exceptional verification of the evaluations is
done on site by nurses not belonging to the institution.
They are generally nurse-instructors-desk-reviewers responsible of the PLAISIR©
system.
|