TV program KPI changes? What is the way to reform the evaluation system

the CCTV column comprehensive evaluation system optimization scheme and annual brand column selection scheme, which was officially implemented a few days ago, has aroused widespread concern in the industry, academia and society.

at the beginning of this century, Chinese TV entered the evaluation era marked by the introduction of CCTV program evaluation system. From the initial comprehensive evaluation system of radio and television programs, to the controversial last elimination mechanism of radio and television programs, and then to the column comprehensive evaluation system recently implemented by CCTV, the evaluation of Chinese TV programs can be described as a sword every ten years. How does this new set of column comprehensive evaluation system operate? What is the essential difference between the evaluation system and the previous evaluation system? What kind of impact will it have on the industry and society? Can it be promoted in provincial TV stations? This issue of Media Weekly tries to take the new evaluation system of CCTV as a starting point to explore the way to reform the evaluation system of TV programs.

< strong > CCTV implements the new evaluation system < / strong >

the "CCTV column comprehensive evaluation system optimization scheme and annual brand column selection scheme" (hereinafter referred to as "optimization scheme") has been implemented, which is constantly causing strong repercussions in the industry. Under what kind of background is this new evaluation system born? What considerations are included in the specific operation process? And how to evaluate the column and select the brand column?

< strong > system focuses on social effect < / strong >

Xu Lijun, deputy director of the market evaluation department of the CCTV editor's office, said in an interview with the China News and Publishing News that the primary goal of CCTV in formulating and issuing the "optimization plan" this time is to improve program quality by selecting annual brand programs, promote CCTV's brand construction, and comprehensively enhance CCTV's guidance, influence, and communication power.

this scheme takes the column as the evaluation object, and together with the channel evaluation system which takes the channel as the evaluation object and the program evaluation system which takes the program as the evaluation object, constitutes the evaluation system of channel & mdash; program & mdash; program & ldquo; trinity & rdquo;. It is understood that this scheme is based on the basic plan of CCTV brand column construction and the central leaders put forward that the evaluation system of ldquo; column should focus on the social effect & the spirit of rdquo;, and finally formed on the basis of the original column evaluation system. Xu Lijun said that the "optimization plan" pays attention to the evaluation of the social effects of programs such as guidance and influence, deepens the investigation of the communication power of programs, and strengthens the judgment of the professionalism of programs, which relies on the evaluation of social effects. an evaluation system that attaches importance to market performance and professional quality considerations.

< strong > set four major evaluation indicators < / strong >

Huang Jinghong, head of the comprehensive evaluation team of the Market Evaluation Department, said that in the course of concrete operation, the "optimization plan" has set up four major indicators, namely, the guiding power index and influence index of the social effect of the evaluation column communication, the communication power index of the evaluation column communication market effect and the development of the column, and the professional index of the professional quality of the evaluation column.

among them, & ldquo; guiding force & the direction of rdquo; assessment communication, characterizing whether the column orientation is correct, whether it is conducive to carrying forward advanced culture, reflecting mainstream value, improving aesthetic taste, leading morals and other indicators of social effect. The index is obtained by the score of the expert evaluation team and the special survey of the audience, accounting for 20% of the total. & ldquo; influence & the depth of communication measured by rdquo;, including two second-level indicators of credibility and satisfaction, which are obtained by the audience survey, accounting for 25% of the total. Among them, credibility characterizes the social effect indicators of the column's credibility, authority and sense of responsibility; satisfaction refers to the score of the audience who saw the column 30 days before the survey. & ldquo; communication power-the breadth of rdquo; evaluation communication, which includes four secondary indicators: audience target completion rate, audience size, loyalty and growth trend, which is calculated from the ratings survey data, accounting for 50% of the total. Among them, the audience target completion rate refers to the ratio of the average rating of the column in the evaluation cycle to the target value of the rating in the evaluation cycle issued by the channel to which it belongs; the audience size reflects the size of the audience in the evaluation cycle; the loyalty index reflects the repeated viewing of the column. The growth trend is to measure the growth of the column by comparing the average ratings of the column during the evaluation cycle with the average ratings of the same period in the previous year. & ldquo; professionalism & the production level and quality of the rdquo; evaluation column is an examination of the proficiency of TV production, accounting for 5% of the total. The index is scored by the expert evaluation team, which mainly includes six consideration dimensions, such as editing, production and editing, broadcast host, sound music, picture shot (dance beauty), text writing and so on.

< strong > implement scientific data collection < / strong >

it is understood that the data sources of all the evaluation indicators in the "Optimization Plan" are divided into three parts, namely, the survey of viewers, the scoring of experts, and the collection of ratings data.

Huang Jinghong said that the comprehensive evaluation system of the column takes quarterly as the implementation cycle, with 12000 samples of audience survey carried out each quarter, which are randomly selected from 184 sampling sites across the country. The overall target is all household televisions in 30 provinces (autonomous regions and municipalities) across the country. In the construction of the expert database, an expert group composed of more than 300 people scored the column, including the relevant leaders of the propaganda Department of the Communist Party of China Central Committee, the Information Office of the State Council, the State Administration of Radio, Film and Television, and other competent departments. influential media authorities, as well as associate professors and above teachers engaged in communication and journalism research in colleges and universities, as well as practitioners in the journalism industry. Experts are graded in the way of online survey by online questionnaire, which places three column videos selected from the evaluation quarter, and sets the limit that you must watch for at least 5 minutes before you can score. In addition, the professional background, age and gender of the experts are taken into account, and the expert groups and column groups are randomly matched to ensure that at least 10 experts are graded by each column. In addition, the data collection source of the communication power index is the ratings survey data provided by CCTV Suofuri, and the communication power indicators are calculated on the basis of the average audience data of three months in the evaluation cycle.

Xu Lijun said: the column comprehensive evaluation system is an important means of macro-management in Taiwan, and the column evaluation results play an important guiding and exemplary role. It is understood that most of the programs of CCTV participate in the comprehensive evaluation and publish the top 30 comprehensive rankings every quarter. The annual brand column is ranked according to the average score of the comprehensive evaluation results of each quarter of the year, ranking among the top 30 finalists of the annual brand column. The comprehensive evaluation results of the column will be used as the main reference condition for the selection of the annual brand column, and the final award-winning columns and personnel will be commended and rewarded.

< strong > New Evaluation system VS Last knockout system < / strong >

for a long time in the past, CCTV has been using the 2002 version of the evaluation system, commonly known as the last elimination system, with & ldquo; three indicators, a ruler & rdquo; as the characteristics of the evaluation system. Speaking of this controversial evaluation system, Xu Lijun said: & the ldquo; evaluation system is a management tool in Taiwan and is subject to the management objectives of different stages. The implementation of the last elimination system has controlled the total number of columns of the whole station to a great extent, eliminated 71 columns in the past 6 years, solved the problem of intermingled good and bad columns and mixed fish and dragons, and realized the original intention of the design in management. & rdquo;

so, what are the new characteristics of the new evaluation system compared with the previous last elimination system? In this regard, Zhang Junchang, secretary general of the China Radio and Television Association, said: first, the index system set by the "optimization plan" is more scientific and perfect, and the four major indicators take into account the social effect, market effect, and professional quality of the program. it not only takes into account the social and economic benefits of the program, but also strengthens the requirements for the quality of the program. This is the development and improvement of the three indicators of & ldquo; and a ruler & rdquo; under the new situation and new environment. The second is to refine each index to make it operable. Third, the "optimization scheme" takes into account the diversity of channels and columns, and treats the programs differently through classified evaluation on the premise of implementing unified evaluation indicators. The fourth is to change the weighted proportion of each index item to make the evaluation more scientific and reasonable. The "optimization scheme" gives weight to guidance, influence, communication and professionalism. Compared with the past, it not only pays more attention to subjective indicators such as guidance, influence and professionalism, but also further weakens the dependence on simple ratings through the fine division of the secondary indicators of communication power, such as audience size, loyalty and growth trend, etc. Fifth, the "optimization scheme" aims at the industry-ldquo; audio-only rate-rdquo; tendency to carry out bold reform, creatively proposed to use the column audience target completion rate index to replace the absolute rating index. This practice of bringing the audience target completion rate into the category of target management has got rid of the stereotype of ldquo; audiovisual rate & rdquo;.

Liu Yannan, director of the Communication University Of China audience Research Center, believes that CCTV's new evaluation system emphasizes the evaluation of the social effects of communication, weakens the market factors slightly, and pays attention to the professional quality of the column. The evaluation dimensions of the audience market effect are relatively diverse, which reflects a certain transformation of the TV man-ldquo; ratings view-rdquo;, at least from the pursuit of audience maximization to scientific rationalization. In addition, the biggest difference between the new evaluation system and the previous last elimination system is that it no longer wields the big stick of elimination, but adopts a ranking incentive mechanism.

< strong > do not engage in ratings-only boycott & ldquo; three vulgarity & the style of rdquo; < / strong >

when it comes to TV program evaluation, people always think of a word & ldquo; ratings & rdquo;. For the use of this foreign word, Chinese TV people and the broad masses of viewers have different responses. On the one hand, it represents the relatively reasonable rules of the market game in program evaluation and even internal management and external operation under the background of the deepening transformation of the TV market at that time; on the other hand, it also brings the malpractice of Chinese TV program market & ldquo; ratings & rdquo; leading to a period of TV programs & ldquo; & rdquo; tendency. It should be said that the implementation of the comprehensive evaluation system of CCTV programs has aroused strong repercussions in society, which to a large extent reflects that the audience pays more attention to the social effects of Chinese TV programs.

Why is the evaluation system with ratings as the main evaluation index so criticized? Xu Lijun said: in fact, few TV stations take ratings as the only evaluation index, and CCTV has never done so. The reason why the audience rating is widely known is that, on the one hand, it is a concrete figure, which can be known every day; on the other hand, the audience rating does occupy too much share in the program evaluation, so it is misunderstood by everyone.

Zhang Junchang holds the same view. He said that although we do not take ratings as the only evaluation criterion, because the ratings account for a very large proportion, when implementing the last elimination system, some elegant things that can nourish the national soul may be eliminated. The main reason for the problem of ldquo; ratings lies in the one-sidedness of the data survey company and the lack of science and objectivity. & rdquo; Zhang Junchang said frankly that he did not object to strong data research companies providing objective and accurate ratings data, but some current research companies did not do this. The data provided had problems such as a small number of sample data and pollution of sample households, and everyone was unable to reverse the sledgehammer of ratings in the market, which finally caused individual TV stations to come up with strategies for the pursuit of ratings.

when it comes to ratings, he Sulu, associate dean of Communication University Of China TV and Journalism School, thinks this is a strange evaluation indicator. Because from the simple figures, it is difficult to see the audience, mentality and reaction after watching the program, high ratings does not represent high quality programs, for those low-quality programs, the higher the ratings, the greater the negative effect on the society.

At the national work conference on cultural system reform held this year, Cai Weichao, deputy director of the propaganda Department of Communist Party of China Central Committee and director of the State Administration of Radio, Film and Television, stressed that we should attach importance to audience ratings, but not ldquo; ratings only-rdquo;, should establish a scientific mechanism for the collection, analysis, and release of viewing data, and study and formulate scientific standards and systems for comprehensive evaluation of programs. In the new evaluation system, CCTV tries to change the practice of relying solely on survey companies, turning ratings into communication power, accounting for only 50% of the weight. Even so, it is not calculated by the absolute value of the ratings, but by the target completion rate. & ldquo; does not just look at the current or this year's data, but compared with previous years, this approach breaks through the problem that the column has been thinking about in the past. & rdquo; Zhang Junchang commented.

Liu Yannan said that the program evaluation system is not only an effect evaluation mechanism, but also an incentive and management mechanism, but also a guiding mechanism. At a time when public opinion is increasingly calling for purifying the screen, advocating healthy and upward social customs, and resisting vulgarity, vulgarity and kitsch, CCTV has launched a new program evaluation system, hoping that ldquo; will change from focusing on ratings to comprehensive evaluation of columns, and consciously practice the socialist core value system & rdquo. Emphasizing the innovation of TV communication in value orientation and management mode is another major effort after the first introduction of the evaluation system. He Sulu believes that today, when the media affects the world, the impact of television media on society is still very great. CCTV's reform of the evaluation system shows its attitude towards TV program orientation, which makes the program development have a clearer direction, which is a good exploration to curb the commercialization of TV programs and lead the trend of the times. Zhang Junchang also said that while the "optimization plan" attaches importance to the scientific rationality of qualitative and quantitative evaluation, it also pays attention to encouraging the innovation and excellence of the columns through the evaluation of programs, driving negative incentives with positive incentives, and driving punishment with awards. The hard elimination column is actively replaced, so as to realize the linkage between the macro-control of the whole station and the fine management of the channel, and also make the construction of the brand column move towards a sound development track.

< strong > cannot be promoted to provincial TV stations across the board < / strong >

it is understood that after the introduction of the new column evaluation system of CCTV, Beijing TV Station has recently introduced a set of brand-new column evaluation system according to its own reality. So, can the evaluation scheme of CCTV be promoted at the provincial local stations? Where should the reform of China's TV program evaluation system go in the future?

Liu Yannan believes that although CCTV's new evaluation system has integrated various dimensions and indicators, and has a comprehensive construction of both humanistic and technical factors, in a sense, it reflects the higher level of analysis and application of audience information in Chinese television circles, but there are also some problems worth discussing. For example, in today's increasingly active new media, how to include new media indicators in the traditional television evaluation system is still a problem. For example, the operation frequency of quarterly program evaluation requires high human, financial and material resources, and it is worth paying attention to whether the new system can run for a long time. Tang Shunrong, director of the editorial department of China Radio, Film and Television, found in an interview that it is of little significance to promote CCTV's approach to provincial television stations, and that is precisely because of this: first, the money for large sample research is more than one million yuan a year; second, it takes a lot of energy to hire experts.

In addition, some people in the industry believe that the effect of selecting the top 30 on a quarterly, half-yearly, and annual basis for guiding the production of front-line programs remains to be examined. He Sulu said that CCTV's approach can enable provincial stations to change their concepts, from passive pandering to active guidance to improve media responsibility and professionalism. However, there is a great difference between CCTV as a national television station and provincial stations, and it remains to be evaluated how the evaluation system can be promoted and whether it can get good results.

& ldquo; to promote an evaluation system should take into account regional differences as well as differences in channels and columns, not across the board. & rdquo; Zhang Junchang believes that the "optimization plan" is more scientific, reasonable and operable than the previous assessment system, and the improvement measures provided for the problems faced by the current program evaluation are constructive, which indicates the future development trend of the radio and television program evaluation system in our country. But at the same time, he also mentioned his own concerns, such as the evaluation of the costs and benefits of ldquo; programs, the evaluation of online audio-visual media and the theoretical construction of academic circles. When the traditional radio and television media implement the cost budget system, the consideration of program input and output lags behind, which is generally investigated through advertising revenue and ratings data, which is not scientific. With the deepening of the reform of radio and television system, the continuous implementation of the separation of production and broadcasting and the development of market-oriented audio-visual media, the input-output ratio of programs needs to be paid more attention. This is an aspect that is difficult to deal with in the construction of program evaluation system & rdquo;.