项目效果。
本案例基于python的flask框架,通过爬虫程序将数据存储在csv文件中,在项目运行时会通过render_template映射出对应的页面,并且触发一个函数,该函数会读取csv文件的数据将之交给echarts渲染 ,echarts将之渲染到页面中。
demo.html
from flask import Flask,render_templateimport pandas as pdapp = Flask(__name__)@app.route("/")def show():data = pd.read_csv('data.csv',encoding='gbk').to_dict(orient="records")return render_template("demo.html",data=data)if __name__ == '__main__':app.run()
movie.py
import requestsimport re#获取页面信息url = "https://movie.douban.com/top250"headers = {"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36 Edg/105.0.1343.27"}r = requests.get(url,headers=headers)r = r.text#解析页面obj = re.compile(r'.*" />(?P.*?).*?' r'(?P.*?)人评价',re.S)#匹配result = obj.finditer(r)f = open("data.csv",mode="a")i = 0f.write("name,value\n")#输入for it in result:f.write(str(i)+","+it.group("name")+","+it.group("num")+'\n')i += 1f.close()
demo.html
*{margin:0;padding:0;}var chartDom = document.getElementById('main');console.log(chartDom);var myChart = echarts.init(chartDom);var option;option = {title: {text: 'top',subtext: 'movie',left: 'center'},tooltip: {trigger: 'item'},legend: {orient: 'vertical',left: 'left'},series: [{name: 'Access From',type: 'pie',radius: '50%',data: {{data|tojson}},emphasis: {itemStyle: {shadowBlur: 10,shadowOffsetX: 0,shadowColor: 'rgba(0, 0, 0, 0.5)'}}}]};myChart.setOption(option);
data.csv
data.csv由movie.py运行得到。
name,value0,肖申克的救赎,29088191,霸王别姬,21475092,阿甘正传,21687933,泰坦尼克号,21993374,这个杀手不太冷,23025795,千与千寻,22523546,美丽人生,13308887,辛德勒的名单,11094868,星际穿越,18452219,盗梦空间,206841310,楚门的世界,171750111,忠犬八公的故事,140330212,海上钢琴师,168320213,三傻大闹宝莱坞,186470214,放牛班的春天,131600015,机器人总动员,132034916,无间道,136892017,疯狂动物城,194472318,控方证人,56352319,大话西游之大圣娶亲,153892120,熔炉,93404521,教父,97135622,触不可及,111797023,当幸福来敲门,152493124,末代皇帝,888701
当然,我更希望爬虫程序是自动加载的,可以将之封装为一个函数,在页面加载时调用它。