Walks

网络安全爱好者

Python3学习系列(十二):Scrapy表单登陆

《Python3学习系列(十二):Scrapy表单登陆》

前言:

下面给大家写一点表单登陆在Scrapy上面应用的知识点,内容不多,不多说了,看看代码应该就可以理解,在代码里面我会尽量给点注释


代码实现:

  • Login_Form.py

这里是对我自己的个人博客进行登录

# -*- coding: utf-8 -*-
import scrapy
from scrapy.http import Request,FormRequest

class LoginFormSpider(scrapy.Spider):
    name = 'Login_Form'
    allowed_domains = ['bywalks.com']
    start_urls = ['http://www.bywalks.com/wp-admin/']

    def parse(self, response):
        content = response.xpath('//*[@id="comment-5"]/div/blockquote/p/text()').extract_first()
        print(content)


    #登陆
    login_url = 'http://www.bywalks.com/wp-login.php'
    
    #重写基类的start_requests方法,最先请求登陆页面
    def start_requests(self):
        yield Request(self.login_url,callback = self.login)

    def login(self,response):
        #我们只需要构建用户名和密码即可,因为from_response会把隐藏表单自动填写进去
        fd = {'log':'XXX','pwd':'XXX'}
        yield FormRequest.from_response(response,formdata = fd,callback = self.parse_login)

    def parse_login(self,response):
        #登陆成功后,爬取我们需要内容
        if "欢迎使用WordPress" in response.text:
            yield from super().start_requests()

  • Login_Form.py

# Obey robots.txt rules
#这里要改为False,就是不遵循robots.txt的规则,因为有些网站会不希望爬虫去爬网站的后台,所以添加了这个规则,如何爬虫看到了这个规则,就不会去爬取
ROBOTSTXT_OBEY = False

参考资料:《精通Scrapy网络爬虫》

转载请注明来自:www.bywalks.com

点赞
  1. I definitely wanted to write down a message so as to thank you for all of the lovely facts you are giving on this site. My incredibly long internet research has at the end been compensated with professional know-how to exchange with my close friends. I 'd say that most of us readers actually are very much blessed to exist in a decent site with very many special professionals with interesting pointers. I feel very much privileged to have come across the web site and look forward to so many more enjoyable minutes reading here. Thank you once again for a lot of things.

  2. My husband and i ended up being so fortunate Louis could finish off his web research by way of the precious recommendations he obtained when using the web pages. It is now and again perplexing to just be handing out information which usually some others could have been trying to sell. And we all recognize we have got the blog owner to appreciate because of that. The type of illustrations you made, the easy web site menu, the friendships you will make it possible to foster - it's mostly fabulous, and it's really leading our son and our family believe that the content is cool, and that is unbelievably pressing. Thanks for all the pieces!

  3. curry 4说道:

    Thanks a lot for giving everyone an exceptionally wonderful chance to read from this site. It can be very good and also jam-packed with fun for me and my office co-workers to search the blog at the least 3 times in 7 days to find out the newest secrets you will have. Not to mention, I'm also certainly satisfied with all the special information served by you. Some 1 ideas in this posting are rather the most effective we've ever had.

  4. reebok shoes说道:

    Thanks so much for providing individuals with remarkably pleasant chance to read from this blog. It is usually very great and as well , stuffed with a lot of fun for me and my office friends to search the blog at least three times in a week to read through the new issues you have. Not to mention, I'm so at all times pleased with your exceptional strategies you serve. Certain 3 tips in this posting are completely the simplest we have all ever had.

  5. foamposites说道:

    Thank you a lot for giving everyone an extraordinarily nice opportunity to read in detail from this site. It really is so enjoyable and also jam-packed with amusement for me and my office peers to visit your blog particularly three times every week to find out the newest issues you have. And indeed, we are certainly impressed with your tremendous points you give. Some 2 tips in this posting are rather the most beneficial I've had.

  6. ultra boost 3.0说道:

    I as well as my guys were viewing the excellent tips and hints from your web page and so at once I had a terrible feeling I had not expressed respect to the web site owner for those techniques. The guys were thrilled to see them and have in effect in reality been enjoying these things. We appreciate you getting considerably helpful and also for making a decision on variety of cool things millions of individuals are really wanting to discover. My personal honest regret for not saying thanks to sooner.

  7. nike huarache说道:

    I am commenting to let you know what a remarkable experience my cousin's daughter developed checking your webblog. She came to understand many things, not to mention what it is like to have a wonderful teaching character to get the rest effortlessly know just exactly a variety of extremely tough matters. You actually surpassed her desires. Thank you for rendering those powerful, trusted, revealing not to mention unique guidance on that topic to Ethel.

  8. moncler jackets说道:

    Thank you for your own hard work on this site. Kim loves conducting investigations and it is easy to see why. A lot of people hear all of the lively tactic you provide informative suggestions by means of your blog and therefore increase participation from the others on the concern plus my simple princess has always been starting to learn a great deal. Take advantage of the remaining portion of the year. You're the one conducting a really great job.

  9. I am commenting to make you know of the remarkable experience my friend's child undergone reading through your web site. She noticed a wide variety of things, which included what it is like to possess an incredible giving nature to make many more without difficulty master certain advanced topics. You really exceeded visitors' expectations. Many thanks for rendering the precious, safe, explanatory and unique tips about the topic to Gloria.

  10. jordan shoes说道:

    I wanted to write a note to be able to say thanks to you for the stunning recommendations you are posting here. My incredibly long internet investigation has now been recognized with beneficial concept to talk about with my family. I 'd repeat that many of us website visitors actually are extremely blessed to live in a fantastic community with many perfect people with useful things. I feel pretty grateful to have discovered your entire website and look forward to tons of more pleasurable minutes reading here. Thank you once more for a lot of things.

  11. I am just writing to let you understand what a outstanding discovery my cousin's girl enjoyed going through yuor web blog. She noticed a lot of issues, not to mention what it's like to possess a great helping spirit to let folks smoothly grasp several grueling issues. You undoubtedly exceeded her expectations. Thank you for delivering such great, trustworthy, revealing and in addition cool thoughts on the topic to Jane.

  12. I together with my buddies came studying the excellent hints found on your site then all of the sudden came up with a horrible suspicion I never expressed respect to the web site owner for them. All of the people had been absolutely stimulated to learn all of them and have in fact been enjoying them. Thanks for turning out to be considerably accommodating and for making a choice on variety of notable ideas most people are really desirous to discover. Our honest regret for not expressing appreciation to you sooner.

  13. nike sneakers说道:

    My wife and i have been so more than happy when Chris could finish up his investigations with the precious recommendations he obtained from your own web page. It is now and again perplexing to just continually be giving away thoughts that many many others might have been trying to sell. So we take into account we've got the blog owner to be grateful to for this. Those explanations you have made, the straightforward website menu, the friendships you can help foster - it's got most remarkable, and it is letting our son and our family know that that issue is brilliant, which is certainly very mandatory. Thanks for the whole thing!

  14. I wish to express thanks to this writer just for bailing me out of this type of issue. After browsing throughout the the web and coming across recommendations which are not helpful, I figured my life was done. Existing without the answers to the issues you have resolved by way of your main review is a critical case, as well as the kind which could have negatively affected my entire career if I hadn't encountered your web blog. Your primary understanding and kindness in dealing with a lot of stuff was important. I am not sure what I would have done if I had not encountered such a subject like this. I'm able to at this moment look forward to my future. Thank you so much for your expert and sensible guide. I won't hesitate to suggest the blog to anybody who would like guide about this subject matter.

  15. I wanted to construct a quick note to say thanks to you for all the lovely guides you are writing at this site. My extended internet search has now been honored with awesome facts and strategies to share with my relatives. I would assert that most of us website visitors are undoubtedly fortunate to exist in a useful network with so many special individuals with insightful basics. I feel very much blessed to have seen the web pages and look forward to plenty of more fabulous times reading here. Thank you once more for all the details.

  16. yeezy boost说道:

    I am glad for commenting to make you understand of the beneficial encounter my cousin's princess developed visiting your site. She came to find too many issues, with the inclusion of what it's like to have a very effective helping spirit to get the rest effortlessly know certain problematic subject areas. You undoubtedly surpassed our own expectations. Thanks for supplying those insightful, healthy, explanatory and as well as unique tips about this topic to Jane.

  17. louboutin shoes说道:

    My spouse and i ended up being so fortunate Edward could complete his investigation through your ideas he gained through your site. It is now and again perplexing to just choose to be offering solutions which other folks have been trying to sell. And we all see we now have the blog owner to give thanks to for this. The specific illustrations you have made, the simple web site menu, the friendships you can assist to promote - it's many fabulous, and it's letting our son in addition to the family do think that subject is enjoyable, and that is truly essential. Thank you for everything!

  18. I enjoy you because of each of your efforts on this website. Ellie enjoys working on internet research and it's obvious why. All of us know all relating to the dynamic method you deliver worthwhile items by means of your blog and therefore attract contribution from website visitors on the area of interest plus my daughter is without question discovering so much. Have fun with the remaining portion of the new year. You're carrying out a splendid job.

  19. I'm writing to let you be aware of of the impressive experience our child encountered viewing your blog. She learned a lot of issues, not to mention what it is like to possess a very effective giving mindset to make others with ease learn about certain complex topics. You truly exceeded her expectations. Thanks for giving the priceless, safe, edifying and even unique thoughts on the topic to Julie.

  20. yeezy boost说道:

    I definitely wanted to post a brief comment so as to say thanks to you for the lovely hints you are writing on this website. My rather long internet lookup has finally been honored with high-quality facts and techniques to write about with my family members. I would admit that many of us readers are undeniably endowed to exist in a great website with so many lovely people with very helpful tips. I feel extremely lucky to have used the webpages and look forward to so many more awesome times reading here. Thanks once more for everything.

  21. ferragamo belts说道:

    Needed to send you a very little remark just to thank you again for these gorgeous techniques you have provided above. This has been strangely generous of people like you to give extensively all many people could possibly have advertised for an electronic book to make some dough on their own, even more so given that you could have done it if you considered necessary. These concepts additionally served as a easy way to be aware that some people have a similar keenness really like my personal own to find out way more on the topic of this issue. I think there are a lot more fun instances up front for many who scan your site.

  22. lacoste polo说道:

    I happen to be writing to let you be aware of what a nice encounter my friend's child undergone checking your web page. She even learned numerous details, most notably how it is like to possess an amazing helping style to make a number of people with no trouble learn about specific tortuous subject matter. You undoubtedly surpassed visitors' desires. Many thanks for showing those informative, safe, educational not to mention cool guidance on your topic to Lizeth.

  23. ultra boost说道:

    I would like to get across my love for your generosity in support of individuals who need guidance on this particular subject matter. Your very own dedication to getting the message throughout was incredibly significant and have without exception made those like me to arrive at their goals. Your interesting recommendations can mean a lot a person like me and even further to my colleagues. Thanks a ton; from everyone of us.

  24. yeezy boost 350说道:

    I precisely wished to appreciate you all over again. I am not sure what I could possibly have created without the entire concepts contributed by you on that area. It became a terrifying scenario for me, nevertheless spending time with a specialised form you solved the issue forced me to leap for fulfillment. Extremely thankful for this help and believe you comprehend what a powerful job you were accomplishing training people through the use of your blog. I'm certain you haven't met any of us.

发表评论

电子邮件地址不会被公开。 必填项已用*标注