Well, well, I have been doing a lot in 2011, but not contributing there much.
I'm going to address few of my peeves in the next posts. I swear. It will be fun. I promise.
Tuesday, February 21, 2012
Thursday, May 20, 2010
Good KPIs
One of the challenges any organization, including drug safety has, is designing good Key Performance Indicators known as KPI.
Any organization must challenge the way things are done (and for what results). Designing good KPIs
1. Objectives: what are we trying to achieve?
1. “Approach” KPIs
- May be more than one indicator for each objective
- Each objective will have strategies on how to achieve them
- used to assess the present state of progress and to suggest an appropriate course of action.
- can be qualitative or quantitative data related to inputs, processes or outputs.
- can be minimal targets, stretch targets or a combination.
This creates a good starting point but it does not provide the separation between various KPIs.
What you need then to do is to categorize the KPIs in various groups:
Organizing KPIs
As a recommended practice, I found useful to get this segmentation in place:
1. “Approach” KPIs
- How do we have/need change things?
- How do we have/need implement things?
3. “Results” KPIs
- What we have/need obtained?
4. “Improvement” KPIs
- What we have/need to improve?
Scaling KPIs
Once everything is in place, I would use on a 1 to 5 Likert-type scale for both the KPI and the actual target.
A Likert-type scale consists of a series of declarative statements. The subject is asked to indicate whether he agrees or disagrees with each statement. Commonly, five options are provided: "strongly agree," "agree," "undecided," "disagree," and "strongly disagree." Other Likert-type scales include four or six steps rather than five, excluding the undecided position.
Scales developed by the Likert method will ordinarily include from six to thirty declarative statements. Some of these statements will be worded in a positive manner and other will be worded in a negative manner. For example, two statements which might be used in a scale to measure attitude toward capital punishment would be, "Capital punishment is nothing but legalized murder," and "Capital punishment gives the murderer just what he deserves." A person who is in favor of capital punishment would be expected to disagree with the first statement but agree with the second statement. Of course, the person opposed to capital punishment would be expected to give opposite responses. The individual responses "strongly agree" through "strongly disagree" are assigned numbers, usually 1-5. In this manner the responses to the various items are quantified and may be summed across statements to give a total score for the individual on the scale. It is necessary, of course, that the assigned numbers are consistent with the meaning of the response. For example, the first statement above could be scored 1-5 and the second statement scored 5-1. In this way a person with a strongly favorable attitude toward capital punishment would receive a score of 10 for these two items while a person strongly opposed to capital punishment would receive a score of 2.
Attitude scores produced by Likert-type scales have been found by Likert and others to yield data which may be analyzed by "normal curve" statistics. Because this type of data is desired by researchers in most cases, the Likert-type scale provides a very useful and relatively uncomplicated method of obtaining data on people's attitudes for persuasion research.
Sunday, March 7, 2010
PV management and every day operations
Somehow, I found this short story to tell a lot about corporate culture and somehow precisely on the somewhat blurry pictures managers in PV have about the day-to-day front line of operations.
A man in a hot air balloon realized he was lost. He reduced altitude and spotted a woman below. He descended a bit more and shouted, “Excuse me, can you help me? I promised a friend I would meet him an hour ago, but I don’t know where I am.”
.
The woman below replied, “You are in a hot air balloon hovering approximately 30 feet above the ground. You are between 40 and 41 degrees north latitude and between 59 and 60 degrees west longitude.”
.
“You must be an engineer,” said the balloonist.
.
” I am,” replied the woman, How did you know?”
.
“Well,” answered the balloonist, “everything you told me is, technically correct, but I have no idea what to make of your information, and the fact is I am still lost. Frankly, you’ve not been much help so far.”
.
The woman below responded, “You must be in Management.”
.
“I am, “replied the balloonist, “but how did you know?”
.
“Well,” said the woman, you don’t know where you are or where you are going. You have risen to where you are due to a large quantity of hot air. You made a promise, which you have no idea how to keep, and you expect people beneath you to solve your problems. The fact is you are in exactly the same position you were in before we met, but now, somehow, it’s my fault”.
.
The woman below replied, “You are in a hot air balloon hovering approximately 30 feet above the ground. You are between 40 and 41 degrees north latitude and between 59 and 60 degrees west longitude.”
.
“You must be an engineer,” said the balloonist.
.
” I am,” replied the woman, How did you know?”
.
“Well,” answered the balloonist, “everything you told me is, technically correct, but I have no idea what to make of your information, and the fact is I am still lost. Frankly, you’ve not been much help so far.”
.
The woman below responded, “You must be in Management.”
.
“I am, “replied the balloonist, “but how did you know?”
.
“Well,” said the woman, you don’t know where you are or where you are going. You have risen to where you are due to a large quantity of hot air. You made a promise, which you have no idea how to keep, and you expect people beneath you to solve your problems. The fact is you are in exactly the same position you were in before we met, but now, somehow, it’s my fault”.
Friday, August 21, 2009
What could go wrong?
When projects start, who knows when they will end? What do you do to go back on tract?
Each organization has its own merits and challenges. One element revealing how the organization deals with its own challenges is project management. I have been probably involved with 20 different PV organizations in the last three years and none has the same balance or approach here.
What is the main trait of the challenges the organization has with its projects?
I precisely set the tip point to taking a decision and not making a decision.
Making a decision is a process with inputs, recommendations etc.
Taking a decision is an action done by either an individual or a group.
That's where most of the organizations fail: they do not know where they really need to go.
As there's no clear heading, there's no precise direction and any road will make it. If it's a group taking the decision, everyone will have a different and somewhat strong opinion. Things will spin out of control since egos are taking over brains.
So how do you go back on track?
You have to be prepared, you have to clearly set the prospective, what a decision will require as preparation, rolling out, changes, risks and benefits. You will have to find ways to monitor and account the results.
The good way to put it, is to start by stating that it's no one fault (if the decision does not have the expected results). When putting in place a governance (giving a code of conduct to people acting under a power or a designated responsible), you need to start as when you were learning to play cards.
All faces up, no one looses or wins, every body learn.
You can do it once or twice.
Once the practice has been set, accounting can start, and as with the cards, the fun will.
Each organization has its own merits and challenges. One element revealing how the organization deals with its own challenges is project management. I have been probably involved with 20 different PV organizations in the last three years and none has the same balance or approach here.
What is the main trait of the challenges the organization has with its projects?
Decision Taking.
I precisely set the tip point to taking a decision and not making a decision.
Making a decision is a process with inputs, recommendations etc.
Taking a decision is an action done by either an individual or a group.
That's where most of the organizations fail: they do not know where they really need to go.
As there's no clear heading, there's no precise direction and any road will make it. If it's a group taking the decision, everyone will have a different and somewhat strong opinion. Things will spin out of control since egos are taking over brains.
So how do you go back on track?
You have to be prepared, you have to clearly set the prospective, what a decision will require as preparation, rolling out, changes, risks and benefits. You will have to find ways to monitor and account the results.
The good way to put it, is to start by stating that it's no one fault (if the decision does not have the expected results). When putting in place a governance (giving a code of conduct to people acting under a power or a designated responsible), you need to start as when you were learning to play cards.
All faces up, no one looses or wins, every body learn.
You can do it once or twice.
Once the practice has been set, accounting can start, and as with the cards, the fun will.
Wednesday, July 29, 2009
Long Time
I have been very busy lately and I did not had time to write here. Shame on me.
Thru those last months, I have faced many times the same situation again: users and managers are loosing the business focus and get an IT-Database approach that is always the low road.
If you want to aim high, pharmacovigilance should always be first about business (as in business process).
As an example, an organization tried to get the events in a case re-ordered after the case was medically evaluate. The reason was they thought using a dedicated procedure capability in their software will help supporting it.
I had to detail why they were wrong. I have not detailed it from an IT perspective (the call they wanted to use was supposed to be a function and not a data updating feature), I took the business road: why is it done AFTER the medical evaluation and not BEFORE?
By putting the discussion on the right level (business) I put them back on the road they should not have left. My approach was acknowledged as the safest (audit anyone?) and the technically meaningful (updating data when you're allowed to).
I hope you will remember that the next time you talk about a software capability in your pharmacovigilance system.
Thru those last months, I have faced many times the same situation again: users and managers are loosing the business focus and get an IT-Database approach that is always the low road.
If you want to aim high, pharmacovigilance should always be first about business (as in business process).
As an example, an organization tried to get the events in a case re-ordered after the case was medically evaluate. The reason was they thought using a dedicated procedure capability in their software will help supporting it.
I had to detail why they were wrong. I have not detailed it from an IT perspective (the call they wanted to use was supposed to be a function and not a data updating feature), I took the business road: why is it done AFTER the medical evaluation and not BEFORE?
By putting the discussion on the right level (business) I put them back on the road they should not have left. My approach was acknowledged as the safest (audit anyone?) and the technically meaningful (updating data when you're allowed to).
I hope you will remember that the next time you talk about a software capability in your pharmacovigilance system.
Tuesday, April 28, 2009
B.3.1e: Interesting case
I was answering today a question on lab test result and I refer to the EMEA documentation to see what was the situation.
As often when I have to look in detail to the guidance, I'm discovering little surprises.
As you might know or not, the European MEdicine Agency is promoting guidance to use at best the electronic transmission of safety information including individual cases.
For that puprose EMEA with other agencies set forth a standard. The standard is divided in various section one of which is E2b (Efficacy guidelines section b of to make it simple).
Each section follows a certain series of updates and the latest one is EV 7.1 (EV stands for Eudravigilance which covvers all the resources involved in monitroing health situation in European Economic Area).
The description of what you should put in the electronic information sent is very detailed and B3.1e (lab test unit) element is mandatory if lab test result (B.3.1d) is provided.
If you fail providing the two fields together, your information is rejected.
The interesting piece is that the authority gives answers thru a working group to the industry now and then (more or less every quater we have updates). Since December 2007, the answer given to one question about B.3.1 element is not compliant with the standard provided by EMEA.
The test is related to Blood Glucose and the result is increased. the working group suggests that you send it as two fields B.3.1c (test name) and B3.1.d (result) without the B3.1.e unit.
I will be amused if one tries to send this to the agency and get the case rejected. If you do, let me know.
Thursday, March 19, 2009
Signal Detection Principles
"Volume 9A, Part I, Section 8 “Overall Pharmacovigilance Evaluation and Safety-Related Regulatory Action” describes the principles of signal detection and highlights the sources of information where a signal may be detected.
Signal detection procedures can range from individual case review, trend analysis of case reports to the use of complex statistical methodologies (e.g. Bayesian methods).
The methods used (including periodicity of review) should be determined, for example, by the product portfolio and the number of reports of suspected adverse reactions received.
The MAH should, based on risk assessment, decide what signal detection methods are to be used for which products or types of products, and should document this risk assessment.
All MAHs are expected to have in place systems and procedures for systematic signal detection that are adequately documented in formalised procedures (e.g. SOPs).
Within this documentation it is useful to provide a definition of what constitutes a potential signal, in order to educate all personnel involved in the process and to identify what requires further evaluation.
These documents should also provide detail about the roles and responsibilities of all personnel involved, the sources of information included in the analysis and the methods used for signal detection.
In addition, formal written procedural documents should describe what actions MAHs intend to take based on the outcomes generated from signal detection activities.
This could include escalation procedures following identification of a potential signal, such as further analysis that will be undertaken or referral to a company committee.
Whilst there are no defined minimum requirements for signal detection, whichever method is employed by MAHs for signal detection, certain criteria should apply, namely:
• The method used is appropriate for the MAH’s data set. For example, the use of complex statistical tools may not be appropriate for MAH’s with a small data set.
Signal detection performed only at the time of PSUR production is unlikely to be adequate in most situations.
However, for a generic substance with low numbers of ADRs (and other safety information) generated which is currently on a six-monthly PSUR cycle, this may be appropriate.
Signal detection procedures can range from individual case review, trend analysis of case reports to the use of complex statistical methodologies (e.g. Bayesian methods).
The methods used (including periodicity of review) should be determined, for example, by the product portfolio and the number of reports of suspected adverse reactions received.
The MAH should, based on risk assessment, decide what signal detection methods are to be used for which products or types of products, and should document this risk assessment.
All MAHs are expected to have in place systems and procedures for systematic signal detection that are adequately documented in formalised procedures (e.g. SOPs).
Within this documentation it is useful to provide a definition of what constitutes a potential signal, in order to educate all personnel involved in the process and to identify what requires further evaluation.
These documents should also provide detail about the roles and responsibilities of all personnel involved, the sources of information included in the analysis and the methods used for signal detection.
In addition, formal written procedural documents should describe what actions MAHs intend to take based on the outcomes generated from signal detection activities.
This could include escalation procedures following identification of a potential signal, such as further analysis that will be undertaken or referral to a company committee.
Whilst there are no defined minimum requirements for signal detection, whichever method is employed by MAHs for signal detection, certain criteria should apply, namely:
• The method used is appropriate for the MAH’s data set. For example, the use of complex statistical tools may not be appropriate for MAH’s with a small data set.
- That data from all appropriate sources are considered.
- MAHs have systems in place to assure the quality of their signal detection processes.
- That any outputs from cumulative data review are assessed by an appropriately qualified person in a timely manner (and that the QPPV is informed of new information relevant to the evaluation of risk/benefit).
- That the MAH takes timely and appropriate actions and decisions based on the outputs from cumulative data review.
- That the MAH adequately documents signal detection and evaluation.
Signal detection performed only at the time of PSUR production is unlikely to be adequate in most situations.
However, for a generic substance with low numbers of ADRs (and other safety information) generated which is currently on a six-monthly PSUR cycle, this may be appropriate.
Subscribe to:
Posts (Atom)