NewStats: 3,263,905 , 8,181,829 topics. Date: Sunday, 08 June 2025 at 05:25 PM 5c2k2x

KVNGQ:

Scrapy

Scrapy doesn't do DOM interactions.

You can skip pop-ups by using selenium to click, another option is to inspect the DOM and see if the content is already loaded there.

KVNGQ:
Please i need assistance on webscraping, i am trying to scrape a web site but a pop up ad keep blocking it. How do i resolve this

what tool are you using for the scraping?

Karleb:


I mean the N + 1 problem in SQL that causes a query to query other queries unnecessarily.

It happens when you have a parent child relationship. You query the db for an object and that objects generate another query for it's children and another for the children's relationship and another e.t.c

This happens often when you are using ORM for your database interactions, and it can be mitigated by the statement. Django's ORM provides the select_related function (it uses underneath) to avoid this problem.

Zabiboy:


No H.F man...
But the question is straight forward...
We're not here to compete...all this explanation would just make a novice think these things are hard....
Scientific computing is not a real life application....it's just like telling a non-programmer that a real life application of python is back-end programming...it wont make sense to him...Simply saying "Python is used to make a website functional" would be better...

That being said, you aren't the only one that has done this tho...i just felt i had to point it out..

None taken . But really it's a real world application for a DS/DA... But got your point nonetheless. GS

Raalsalghul:


Can you please tell me the application of "Numpy" in real life?

Numpy is used for effective array manipulation in place of Python List. Neural networks are built on arrays or correctly giant Matrixes. An operation like dotproduct matmul on numpy is about 140 times faster than using ordinary Python List.
Basically it's a library for scientific computing.

Dum20:
Hello Gurus in the house. I need your help here.

I am trying to work Git Bash to run python commands

I already have Git installed but when i ran these commands

echo 'export PATH="$PATH:/c/s/Nelly/anaconda3:/c/s/Nelly/anaconda3/Scripts"' >> .bashrc

echo 'alias python="winpty python.exe"' >> .bashrc


I get the errors below:

bash: export: `=': not a valid identifier
bash: export: `:/c/s/Nelly/anaconda3:/c/s/Nelly/anaconda3/Scripts': not
a valid identifier

Whenever i launch Git Bash, i get the error below:


bash: export: `=': not a valid identifier
bash: export: `:/c/s/Nelly/anaconda3:/c/s/Nelly/anaconda3/Scripts': not
a valid identifier
bash: alias: python: not found
bash: alias: =: not found
bash: alias: winpty python.exe: not found

Nelly@Nelly MINGW64 ~
$


What could be the problem?

I have searched stackoverflow, but the solutions i saw there did not help me to resolve the fault.

I am wondering if anyone has experienced it

When you echo export, you don't need to surround your address with another quote. The first one before export will do and make sure you close it.

And to avoid errors why not edit the bashrc directly with a text editor and remove the need for echo. Your bashrc will be in your directory.

shegzhkn:


Tried that my friend, thanks i appreciate your input.
Gracias

Ok. This library could also be of help if you need something similar to Java's overloading. https://github.com/mrocklin/multipledispatch

shegzhkn:


How ?, elaborate.

Most of the time **kwargs will be sufficient, and if you need overloading based on compile time differentiation of objects, read about python single dispatch on Google.

shegzhkn:
Python is an object oriented programming language along side language like Java, C++, C#, etc

one of the concept of OOP is polymorphism allowing object to be created in different form.

Now here is the real deal, Java Constructor overloading the same for C# but you have to be creative to do that in python say use of default value in the init method.

But i needed to create an object of a class in different form and using default value will not cut it or make it work the way i wanted it to.

So can I create another class with similar logic with slight discrepancies from the former and i am considering efficiency here.

Thanks

Python s polymorphism very well (almost), could you post a snippet of what you are trying to do in java or elaborate further.

tensazangetsu20:
I have seen a lot of web developers portfolio on twitter and one thing that stands out mostly is the to do list and the calculator. Most web developers seem to have this on their portfolio websites?

Do this projects make one employable. I tried doing the to do app and it took like 8 lines of Javascript. Is it wise to include something so basic on ones portfolio?

I think todos most of the time are evidence of completing a beginner class or a boot camp. Almost every developer possess that so it doesn't really count.

Calculator on the other is different in that there are different approach of coding it. If Else style falls in the to-do category, but a more complex lexing and parsing style where essentially what you are building is a mini compiler is definitely worth to be put on a portfolio.

TheGenius001:
Good day Pals. Please I have been trying to install Django in my system since yesterday but it's been futile.

I mean, I have used my sytem command prompt, Anaconda prompt, pycharm command prompt by typing

"pip install django"

but they all return the same message that reads:

"Could not find a version that satisfies the required django(from version: )
No matching distribution found for django."


Please what could be the problem
I already have installed Python, Anaconda which has Jupyter notebook, I also have pycharm.

Please someone should help a brother profer a solution.

Or is Django something I have to separately before installing it??

Try
conda install django

illicit:


Boko Haram and Nigerian Army

Okay. First BH boys are not unstoppable. They only appear to be due to the incompetency forced on the Nigerian army by their sinister leaders and politician. Nigerian Army also aren't immovable in the long run if they continue on their current path of leaking secret to BH, soldiers reg e.t.c

Nigerian Army should be the unstoppable force thrashing all forms of immovable insurgency.

illicit:

so BH can defeat NA

BH and NA? What are they?

Waiting for the 150m hop test. Thats the real deal.

The immovable object will make way for the unstoppable force by disintegrating provided the object is only immovable and not undestroyable. With persistance, a stream will wash away a mountain.

Nice. Seems like Pluto and planet X didn't make your list.

xoftcore:

I can't believe this
Our satelite just barely left jupiter out of the solar system
and you telling me humans discoverd what seem to be an imagination

The picture is from the oldest light ever picked by hubble. It came from a few million years after the big bang.

Those vehicles were probably doctored by individuals. We should stop wondering about our place in the cosmos and start fixing our earthly probems.

Dum20:
Hello Guys,

As i mentioned earlier. Below is another one from fiverr. The budget is $108. My objective is purely for us to discuss and for people to see real life data science problems that people pay for to be solved for them

I do not even know how to solve this one. I hope out gurus in the house can use it to teach us. Just give us an idea of how to approach it.


>>>>> hi Everybody, I looking for someone for Machine Learning sequence prediction models who are not afraid from numbers & big numbers, So the work will be analyse a file with more then 32000 results of raffles. Each raffle is include 4 cards, Heart, Diamond, Clubs and Leaf. So in each raffle there are result for each cards, means 4 results. Now I think that on this file there is something that return and based on old results new result is coming. There is not pattern or something, you should find something like that. After we success to predict the next raffle, I need it automate, so every time I will load the new CSV it will predict the next results. Do you think you can handle this big file and try to found some algorithm or solution to get high accuracy of the results in next raffle? I've attached here the CSV file with the results. I also know the next raffle result, so let me know if you can predict the results. What I need from you is a "demo" or predict the next raffle results and we compare the results with the real results. Regards.<<<<<

The problem is about sequence predictions. You take an ngram of raffles and you predict the next one. Since you have four classes for each raffle, IMHO you can have four models for each card i.e a model handles all diamonds card history etc.

Once you are done with training the four models, you can then create a function that abstracts the models, takes four cards, distribute them to the models and outputs the predictions.

saheedniyi22:


What do you mean by Dom??

Document Object Model. That's where the browser keeps the HTML tree.

.

saheedniyi22:


Thanks for this,

I have been trying to scrape a certain car selling website (not like nairaland that load all its details once) but the website is such that as you scroll down it keeps bringing information for new cars. When I tried to scrape it, it only gave me the information for 10 cars alone.
Is there a way to do it that it will give me more cars information.

For pages injected with JavaScript, Selenium is the main tool here. You will need to combine Selenium with infinite scrolling (it will stop once it reaches end of page). A good approach is to scroll to the end of the page first and once everything is loaded in the DOM you can grab them and send them to BeautifulSoup for parsing.

lymelyte:
the circled file is the csv file

There is a prepended number in the name of the file with a space, try and edit it out by renaming the file. Then copy the name into pandas.

lymelyte:
Please check out this double dot csv.is it right?

To see where the jupyter kernel is looking for files

execute %ls #the percentage should be there.
and %pwd

this should give you an idea of your active files that python is seeing, and the current directory python is scanning.

         

          DAramis:

Thanks for the effort. I did it but didn't work. You can write the full code for me to have an idea if my request is not to much.
Below is the code:

secretName = 'Joe'
name = input ('What is your name?')
while name != secretName:
    name = input ('What is your name?')
    break
    secret = 'swordfish'
     input ('What is your ?')
    while  != secret:
         = input ('What is your ?')
         

secretName = 'Joe'
name = input ('What is your name?')
while name != secretName:
    name = input ('What is your name?')

secret = 'swordfish'
 = input (f'Hello, {secretName}. What is the ? (It is a fish.)')
while  != secret:
     = input (f'Hello, {secretName}. What is the ? (It is a fish.')
Print(' granted')

ThePlainTruth:

This is my new handle.
The task below which I posted before is what is relevant to this thread and it is a real world business requirement.

 [Write a program that will allow me from the program you write to execute another program on a remote computer and when the program has completed on the remote computer the program you write should be able to accept a return code from the program on the remote computer.

Love this question. As a lazy programmer am going to use Python as there is a lot of abstraction with subprocess handling, and handling SSH2 on Python too is less painful.

Assuming both systems have their cryptographic keys, the script/program will be executed on the remote through the ssh driven terminal, and this will always return predefined return codes upon task completion.

DAramis:
Hello bosses and Madams,
Someone should help me out in this python program (I justed started learning it but was stuck in this assignment). The question is in red while my answer is in blue.

Who are you?
I'm fine, thanks. Who are you?
Who are you?
Joe
Hello, Joe. What is the ? (It is a fish.)
Mary
Who are you?
Joe
Hello, Joe. What is the ? (It is a fish.)
swordfish
Access granted.

My code (though not running)

name = ''
While not name:
print ('Who are you?')
name = input ()
while name != 'Joe':
print ('Who are you?')
break
print ('Hello Joe. What is the ?' (it is fish))
= input ()
while != 'swordfish'
print ('Hello Joe. what is the ?' (it is fish))
break
print ('Access granted')

Your first two loops are redundant. Try and merge them into something like

secret_name = "Joe"
name = input('what is your name')

while name != secret_name:
name = input('what is your name')

Then do the same for the . It's called sentinel controlled loop.

olioxx:

.
Very VERY good write up.
Most people fail to realise that the theory you learn during your Computer Science class goes a long way in furnishing your competency. I am still a CS student and I'm proud of that and I know it is not a waste of time.
You can never compare a CS student to someone who went to a bootcamp. They just dont compare. And to those Self taught coders that exaggerate and that think they are far better than CS student, OYO is your case. A lot of master invention are product birthed by CS students themselve. CS student establish the engine/core while self taught developer build on the engine. Your knowledge of some CS stuff along the years will make you a better programmer in any field of your choice.
Rob Pike co-creator of Go, UTF-8 was a CS student.
Dennis Ritchie co-creator C prog lang, co-creator of UNIX OS, co-creator of B prog lang was a CS student.
Ken Thompson creator of B programming lang,co-creator C prog lang, co-creator of Go, co-creator of UTF-8 was a CS student he is known for UNIX OS that is one of the foundation of modern computing.
Sergery Brin co-founder of Google and X was a CS student.
.
To every self-taught developer(that might think CS studying is a waste of time and effort), I want us play to a little game.
* Explain the internal workings of a compiler, and the possible phases a compiler can have.

* Explain briefly the Architecture of the CLR (Common Language Runtime), JIT(Just In Time) Compilation.
* What three core operation does a basic compiler work with.
* Write a simple Fibonacci series algorithm, where it will print out the series to the 20th term and where you can specify the nth term you desire.
* Write a simple algorithm that can give out the ASCII values you inputted
* Find the 2's complement of 11111000, 11001010, 6A3D, E9F
* As related to x86 architecture find the bit and hexadecimal of the following IP address 127.0.0.1, 956.56.0.0(use little endianess) .
* Briefly explain the internal working of cdecl, fastcall, stdcall. And which does Windows use for callng DLLs and APIs
* List out the possible components that enables Win32 API to access the Kernel Mode of a basics Windows NT Operating system, Briefly explain Native API as related to Windows OS.
.
Note this is for FUN.

what a show off. abeg no kill us please.

shadeyinka:

The issue is that a vast majority of CS graduates are completely useless when it comes to problem solving and practicals except those who develop themself outside the Nigerian school curriculum.

One expects to see CS graduates making waves in solution provision but alas, self taught (including self taught CS graduates) are the king.

CS curriculum is too abstract and useless for now in Nigeria. It is one thing to understand data structures and OS, it is completely another thing to know how to IMPLEMENT the theories


Cc: Corporate2020

You have good points. It's an open secret that the education system is broken. But the thing is other countries also produces theoretical CS graduates even though they have better curriculum, teaching materials and better labs(not talking about laboratory here) but they mitigate the issue by one single factor which is quality internship.

Imagine a CS student in US looking for internship, he has companies like Intel, Nvidia e.t.c to apply if he is into us, or if he is into CNS Cisco and co are there, if he wants system engineering or programming then Google, Amazon, Microsoft are there e.t.c. All major countries provides quality internship to undergraduates but that's not the case in Nigeria and that's why I think we have self taught computer scientist graduates in Nigeria.

Grandlord:
There's room for everyone. The CS graduate specialising in software engineering, is responsible for the high level thinking process required for software design, algorithms, architecture and planning. They then hand down this design and plan to the programmers who do the coding with the programming language of their choice or of the company's choice. Everyone knows their place.

That's the thing. Everybody has a place and everyone is equally important. Bill Gates is a great computer scientist but he managed to fu*ck windows GUI. Imagine he had a great UI/UX designer with him. Steve jobs was a badass GUI guy but Bill Gates with his coding skills was pivotal in the success of the Macintosh